acceptodds
Under review as a conference paper at ICLR 2027

Better than Just Average: Computing and Using Shapley Variance

Abstract

Shapley values summarize a player’s marginal contributions across coalitions by their mean, potentially hiding large, inconsistent effects behind a constant attribution. Shapley variance complements this mean by measuring the squared variation of marginal contributions around it. Despite this natural interpretation, Shapley variance remains rarely used in explainable AI, in part because tools for computing it are limited. We develop exact and efficient algorithms for games represented by sparse polynomials, tree ensembles, and Gaussian processes with product-Hamming kernels. For general games, we build on recent surrogate-based approaches to Shapley value estimation, using our exact algorithms on learned surrogates to efficiently estimate Shapley variance. On local explanation games, for example, our polynomial estimator achieves 16-fold lower median squared error than the prior Monte Carlo estimator. Beyond enabling computation, our work demonstrates how Shapley variance also provides a new lens on Shapley value estimation itself. We show that complementary paired sampling reduces the error of Monte Carlo estimation exactly when the game’s odd component has less total Shapley variance than its even component, and derive a corresponding residual decomposition for Kernel SHAP and Leverage SHAP. Together, these results make Shapley variance practical to compute and reveal how variation in marginal contributions governs the behavior of widely used Shapley value estimators. We hope our work enables explanations that characterize not only the mean marginal contribution but also more accurately summarizes their distribution.

Primary area:
interpretability and explainable AI
Keywords:
Explainable AI, Shapley values, Shapley value estimators

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Xg8aOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper gives methods to compute or estimate the variance of Shapley values. The importance in my own words and understanding is that the Shapley value is an average over coalitions and does not reflect if the attribute's contribution very strongly depends on, or interacts with, other attributes.

Exact algorithms are given to three classes of games, sparse polynomials (Fourier-like), trees, and Gaussian processes. For the general case, heuristics are given by using the above three possibilities as surrogates. The theoretical complexity is also analyzed and shown to be intractable.

I was searching for the literature for good illustrative examples on the use of the variance similar to that of the Shapley value but did not find good examples, which may partly due because prior to this result, no good methods were available to compute the variance. I like the examples provided in Appendix 1.

Critical strengths

S1 Providing efficient computational, exact or heuristic surrogate based, methods for computing the variance of Shapley values, a computationally intractable task. Experimentally showing the efficiency and accuracy compared to existing methods. Source code is available.

S2 Using the variance, giving theoretical insight into why paired sampling heuristic for Shapley values may work.

S3 Good illustrative examples

Critical weaknesses

W1 Limited motivation. The concept, although have already appeared in the literature, was less motivated by examples of usage. Beyond the general understanding that it is good to know if an attribute's contribution widely varies across coalitions, what is the insight? Should one separate attributes into groups in this case, build additive models, for example?

W2 Not independently of W1, I could not understand the examples in Fig 1, see Questions.

Questions for authors

Can you give illustrative examples on how to use the variance of the Shapley value for model explanations in the same way as motivated for the value itself? See W1.

Can you extend Fig 1? For the Housing example, I see no difference in the Shapley value and the Shapley + Variance charts, values on bar labels are the same, what does it mean? For the cat example, what does it mean that Patch B interacts with other patches, can you give visual examples?

2 comments

We thank the reviewer for their time and helpful suggestions. We appreciate their recognition of our methods' “efficiency and accuracy” and the “theoretical insight” our analysis gives into paired sampling.

Beyond the general understanding that it is good to know if an attribute's contribution widely varies across coalitions, what is the insight? Should one separate attributes into groups in this case, build additive models, for example?

Shapley variance can help us decide how much to trust an average contribution and where to look for interactions.

Low variance means the Shapley value represents the player's contribution across most coalitions under the Shapley distribution. This is particularly useful for a player with a near-zero Shapley value: low variance means its contribution is usually small, whereas high variance means the average can conceal substantial positive and negative contributions.

High variance gives us a reason to investigate interactions involving that player. We now lead Figure 1 with a text example: “The acting is good, but the plot is not great.” Both “good” and “great” receive similar Shapley values, even though “great” is a stronger positive word modified by “not.” Its higher variance makes this difference visible and motivates examining how its contribution changes when “not” is present.

We added these uses to the introduction. Evaluating whether variance can guide more efficient interaction analysis is a natural next step that our computation methods make possible.

For the Housing example, I see no difference in the Shapley value and the Shapley + Variance charts, values on bar labels are the same, what does it mean?

We have revised Figure 1 to make it more clear: The first column shows the Shapley value explanations while the second column shows the same explanations with Shapley variance highlights. In particular, the thickness of the purple border around each player indicates the magnitude of its Shapley variance.

In the housing example, rooms and bedrooms have similar Shapley values, but bedrooms contribute positively on average without floor area and negatively with it. In contrast, rooms contribute negatively in every coalition. The thicker border around bedrooms shows their higher Shapley variance, revealing a difference that the similar Shapley values conceal.

For the cat example, what does it mean that Patch B interacts with other patches, can you give visual examples?

We meant that a region's contribution depends on which other regions are visible, but the cat example did not show this clearly. We have replaced it with a sea-slug image: Body region A and backdrop region B have similar Shapley values, but A has much higher Shapley variance. A contributes more on average when adjacent body region C is visible, while B changes much less.

Reviewer_Xg8aOpenReview ↗

Maintaining my support of acceptance

I am satisfied with the answers to my as well as to other reviews. I maintain my already positive recommendation.

Reviewer_45WnOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the efficient computation and estimation of Shapley variance, which measures the variability of marginal contributions across coalitions. For structured games, the authors derive exact variance expressions and computational procedures based on polynomial representations, tree ensembles, and GP with product-Hamming kernels. For general games, they propose sigma-SHAP, a surrogate estimation approach that combines the exact variance of a fitted structured surrogate with covariance and residual corrections. The paper also builds on the odd/even decomposition studied in OddSHAP to characterize when complementary paired sampling reduces Shapley-value estimation error for both Monte Carlo and weighted least-squares estimators. Experiments across five application domains compare the proposed variance estimators against Monte Carlo and fractional factorial design baselines, showing lower nMSE in various cases.

Critical strengths

Studying the variability of marginal contributions is a meaningful direction for complementing conventional Shapley explanations, and efficient variance estimation could facilitate its practical application.

  • Exact computation for structured games. The paper provides exact methods for computing Shapley variance using polynomial representations, tree ensembles, and Gaussian processes with product-Hamming kernels, building on existing techniques in the Shapley literature. These results establish a useful connection between existing computational approaches and the analysis of marginal contribution variability.
  • General game estimation with empirical improvements. The proposed sigma-SHAP extends these computational methods to general games by fitting structured surrogates and estimating the remaining covariance and residual variance terms. Experiments across five application domains demonstrate improvements in median nMSE over Monte Carlo and fractional factorial design baselines (Table 1), demonstrating the practical potential of the approach.
Critical weaknesses
  1. Insufficient articulation of the core technical contributions and their alignment with the motivation.
  • Technical contributions. Although Sections 3–5 present several theoretical and computational results, the manuscript does not sufficiently articulate the central technical challenges addressed or the substantive insights gained. In Sections 3 and 4, it remains unclear which steps in extending existing Shapley computation and surrogate estimation techniques to variance are technically nontrivial, and what new insights the resulting formulations provide. Similarly, Section 5 derives conditions for paired sampling but provides limited discussion of their additional implications beyond the odd/even decomposition established in OddSHAP. Consequently, the manuscript does not sufficiently establish which technical advances are nontrivial relative to prior work or why their implications are significant.
  • Motivation–contribution alignment. The title, Introduction, and Figure 1 emphasize the explanatory value of Shapley variance in revealing context-dependent effects obscured by averaged marginal contributions, a problem already examined in prior work but not sufficiently distinguished from the paper's own contributions (see Q2). However, the main theoretical results and empirical evaluations focus primarily on variance computation and estimation. The manuscript does not clearly separate this established explanatory motivation from its computational advances or explain how the two are connected. As a result, the presentation blurs the distinction between the motivating problem and the paper's novel contributions, making their scope and broader significance difficult to assess.
  1. Insufficient analysis of the reliability and limitations of the central estimation approach.
  • A central practical contribution is surrogate-based Shapley variance estimation with residual corrections (sigma-SHAP). However, Appendix E.4 documents severe failures of the earlier GP estimator, where large surrogate errors on unobserved coalitions were not adequately captured by corrections using the fitting samples. Although the revised GP substantially mitigates these failures, the implications for the reliability of the residual correction approach receive little discussion in the main text, including whether similar risks arise with other surrogate families. Moreover, some experiments retain the earlier GP implementation rather than evaluating the final method consistently. Given that practical estimation performance is a principal contribution, the limited analysis of these failure modes and the incomplete evaluation of the final GP estimator weaken confidence in the approach's broader applicability.

(Minor typo) Line 423: “Equation equation 15” → “Equation 15.”

Questions for authors

Q1. What are the main nontrivial technical advances beyond existing computational and estimation methods?

  • Sections 3 and 4 extend established techniques to Shapley variance, but the manuscript does not sufficiently clarify which steps constitute nontrivial technical advances. Specifically, Algorithm 1 relies on an existing Fourier representations of tree (Gorji et al., 2025), Theorem 3.5 applies established quadrature techniques to pairs of GP kernel terms (Mohammadi et al., 2026; Wettenstein et al., 2026), and Section 4 uses a standard variance decomposition with surrogate models using residual corrections (Witter et al., 2025; Thies et al., 2026). Could the authors clarify the main technical challenges in extending these methods from first to second moments, how their approaches overcome these challenges, and what new guarantees distinguish their results from prior work? A clearer explanation of these connections would help assess the technical novelty and significance of the contributions.

Q2. How does the explanatory contribution of this work differ from prior studies of interaction-related limitations in Shapley explanations?

  • The title, Introduction and Figure 1 motivate Shapley variance by showing that averaging marginal contributions can obscure context-dependent effects and interactions. Related explanatory limitations have also been studied in Huang & Marques-Silva (2024) and Chang et al. (2025), with the latter analyzing how negative interactions can lead to underestimated Shapley attributions. More directly, Chang et al. (2025, ICLR 2026 submission, Sections 4–5) study context-dependent interaction effects beyond individual marginal contributions, illustrating how similar expected effects can mask different context-dependent behaviors and motivating variance as a diagnostic for further higher-order analysis. Given this overlap in the explanatory motivation, could the authors clarify what additional explanatory insights or empirical evidence this work provides beyond these prior studies? If the primary contribution is the efficient computation and estimation of Shapley variance, I would suggest positioning the paper more explicitly around this computational contribution and distinguishing it from the established explanatory motivation.
    • Huang & Marques-Silva. "On the failings of Shapley values for explainability." (2024)
    • Chang et al. "Rethinking shapley value for negative interactions in non-convex games." (2025)
    • Chang et al. “Analysis of High-order Interactions in Shapley value for Model Interpretation.” (2025)

Q3. What additional insights or practical implications do the paired-sampling conditions provide beyond OddSHAP?

  • Section 5 characterizes when paired sampling improves Shapley value estimation. However, particularly for Monte Carlo estimation, the condition in Corollary 5.2 appears to follow rather directly from the odd/even decomposition established in OddSHAP (Fumagalli et al., 2026). While the resulting conditions are mathematically valid, their additional implications beyond the existing understanding of paired sampling are not sufficiently discussed. Moreover, it remains unclear how these conditions can be used together with the Shapley variance estimators proposed in Section 4 to determine when pairing is beneficial. Could the authors clarify the new insights provided by these results and their connection to the proposed estimation methods?

Q4. What are the reliability and limitations of the proposed residual-correction approach?

  • Section 4 proposes a surrogate-based variance estimator with residual corrections, but the reliability and limitations of these corrections are not sufficiently discussed. In particular, Appendix E.4 documents severe GP estimation failures where substantial errors on unobserved coalitions were not adequately captured by corrections using the fitting samples. Although the revised GP substantially mitigates these failures, their broader implications for the proposed estimation framework receive little attention in the main text. Could the authors clarify under what conditions these corrections can be considered reliable, whether similar issues may arise with polynomial and tree surrogates, and how such failures might be detected or mitigated? Additionally, the reported GP experiments use different fitting implementations, leaving the evaluation of the final estimator incomplete.
3 comments
AuthorsOpenReview ↗

General Rebuttal

We thank the reviewer for their time and detailed feedback. We appreciate their view that studying marginal contribution variability is “a meaningful direction” and their recognition of the “practical potential of the approach”.

CW 1(a) / Q1

unclear which steps in extending existing Shapley computation and surrogate estimation techniques to variance are technically nontrivial [...] What are the main nontrivial technical advances beyond existing computational and estimation methods?

The main challenge in our work is that Shapley values are linear in the game, while Shapley variance is quadratic. While methods for Shapley values compute the average marginal contribution, for variance, we also need the average squared marginal contribution. When a game is expressed as a sum of terms, this requires accounting for how those terms vary together.

We added a paragraph to the preliminaries to explain the existing surrogate approach and where our extension begins.

Our proofs use variance identities, cancellation of cross terms between odd and even components, and existing polynomial and quadrature methods. To clarify our contribution, we have added a description before each result that describes both the most closely related prior result and the proof techniques we use. We also include a table below to summarize the tools used in each proof, and compare to prior work.

At a high level, we would like to emphasize that we do not claim (or believe) that our work builds new technical tools. Instead, we view our contributions as laying the groundwork for efficiently computing Shapley variance and demonstrating how understanding Shapley variance is useful (e.g., characterizing when paired sampling helps Shapley value estimation).

CW (b) / Q2

If the primary contribution is the efficient computation and estimation of Shapley variance, I would suggest positioning the paper more explicitly around this computational contribution and distinguishing it from the established explanatory motivation.

We agree with this recommendation: Before we can investigate where Shapley variance is useful, we need practical ways to compute it. This is the foundation we want to provide, and we have revised the contribution statement to make that focus clearer.

We have also revised the related-work section to explain how earlier papers address the limitations of averaged contributions and clarify our focus on computing Shapley variance:

Several works examine what Shapley values miss when explaining a model. Huang & Marques-Silva (2024) show that Shapley scores can disagree with formal notions of feature relevance, and discuss alternatives based on formal explanations. Chang et al. (2025) study how negative interactions can undervalue players and propose an allocation rule that keeps only positive interactions. Kumar et al. (2021) introduce Shapley residuals to measure interaction information that Shapley values leave out. We share the motivation of looking beyond a player's average contribution. Our focus is on Shapley variance, and in particular on making it efficient to compute.

CW 2 / Q4

What are the reliability and limitations of the proposed residual-correction approach? [...] Additionally, the reported GP experiments use different fitting implementations, leaving the evaluation of the final estimator incomplete.

Our earlier Gaussian-process (GP) fit sometimes learned a very large intercept and produced inaccurate variance estimates. We discovered this problem by comparing the final estimates with exact Shapley variances. Those comparisons led us to investigate the GP fit and correct it by fixing the intercept to the sample mean during fitting.

The reviewer is right that we should have applied this correction consistently throughout the experiments. We have now rerun all reported GP experiments with the corrected fit: Every revised GP table and figure uses the final implementation.

More broadly, our evidence for reliability comes from the ground-truth experiments. For all three surrogate families, we compare the final variance estimates with exact Shapley variances, so we evaluate the complete algorithm, including its correction terms. The earlier GP failure shows why this matters: the comparison exposed an error that we could then diagnose and correct. We use the same evaluation for our polynomial, GP, and tree estimators across all the reported applications. The revised results demonstrate that the corrected algorithms reliably estimate Shapley variance in these experiments.

Presentation: Difficult to follow despite careful reading.

We added a short explanation of surrogate-based Shapley value estimation in the preliminaries, clarified where our methods extend prior work, and described the tools used in each proof before stating the result. We have marked these revisions in blue. We have also corrected the duplicated “Equation” typo.

We answer Q3 below.

AuthorsOpenReview ↗

Significance of Paired Sampling Result

Q3

the condition in Corollary 5.2 appears to follow rather directly from the odd/even decomposition established in OddSHAP [...] Could the authors clarify the new insights provided by these results and their connection to the proposed estimation methods?

We agree that the Monte Carlo condition is straightforward in hindsight: Once we write the expected error in terms of odd and even Shapley variance, comparing paired and unpaired sampling takes only a few lines. Yet, we think there is value in simple explanations of unexplained phenomenon.

For example, the benefits of pairing and their connection to odd and even components also seem natural in hindsight, but these ideas developed across many years. Castro, Gómez & Tejada studied Monte Carlo estimation of Shapley values in 2009. Over a decade later in 2022, Mitchell, Cooper, Frank & Holmes used paired sampling to (sometimes) make the Monte Carlo estimator more accurate. In 2026, Fumagalli, Butler, Kang, Ramchandran & Witter explained why paired sampling can sometimes help: They showed how pairing separates the odd and even parts of a function, and made the connection that Shapley values only depend on the odd part of the function. Today, all these results about the Monte Carlo estimator can be described in half a page but it took 17 years to find them.

We view our work as the culmination of this story: OddSHAP explains why pairing can help, but its decomposition does not determine exactly when the benefit outweighs the cost of having fewer independent samples. Notably, the paper shows cases where paired sampling can help or hurt the variance of estimators but does not explain why. Our variance perspective makes this tradeoff explicit for the Monte Carlo estimator (Corollary 5.2), and regression-based estimators (Theorem 5.3). Even though the Monte Carlo result is not technically challenging, it finally explains exactly when paired sampling helps. And we see this result as evidence for why studying the Shapley variance is worthwhile :)

In terms of the connection to our algorithms, our proposed estimators can estimate the component variances needed for the Monte Carlo comparison. We leave developing and evaluating additional applications of Shapley variance, including using these estimates to decide whether to pair, to future work.

AuthorsOpenReview ↗

Summary of Proof Techniques

What are the main nontrivial technical advances beyond existing computational and estimation methods?

In addition to short descriptions of proof techniques and prior work added before each result in the paper, we summarize the main-body results below.

ResultPrior workOur work
Lemma 3.1 (Fourier Representation of Shapley Variance)[GMR, MOR '00] recover Shapley values from Banzhaf interactions, which are rescaled Fourier coefficients.We expand the variance identity in the Fourier basis; cross terms between odd and even degrees cancel.
Lemma 3.2 (Möbius Representation of Shapley Variance)[Ortmann, '16] computes Shapley variance for quadratic games. [Kojadinovic, EJOR '07] gives a related Möbius quadratic form aggregated across players.We give each player's variance for an arbitrary Möbius expansion by averaging products of terms.
Lemma 3.4 (Exact Shapley Variance for Tree Ensembles)[KM, SICOMP '93] derive Fourier coefficients from trees; [GAK, NeurIPS '25] use them to compute Shapley values.We combine this conversion with Lemma 3.1 to compute ensemble variance, including covariance between trees.
Theorem 3.5 (Exact Shapley Variance for a Fitted GP Mean)[RFMBF, ICML '26] use GP surrogates; [MRSMC, '26] and [WMY, '26] compute Shapley values by quadrature using Owen's integral [Owen, MS '72].We integrate products of kernel terms to compute the expected squared marginal contribution, then subtract the squared Shapley value.
Lemma 5.1 (Monte Carlo Shapley Value Error)[HM, PCPS '56] give the negative-covariance condition for pairing; [MCFH, JMLR '22] apply it to Shapley estimation. Neither expresses it in odd/even Shapley variances.We express this covariance through odd/even Shapley variances: at equal marginal-sample budgets, pairing helps exactly when the summed even variance exceeds the summed odd variance.
Theorem 5.3 (Regression Shapley Value Error)[FBKRW, ICML '26] separate odd/even objectives for fixed paired samples. [MW, '25] analyze asymptotic error with equal base draws, hence twice as many paired evaluations.We compare finite-sample expected errors at equal coalition budgets: unpaired odd/even errors add; pairing removes even error but fits the odd component with half as many independent coalitions.

Again, we would like to emphasize that we do not claim (or believe) that our work builds new technical tools. Instead, we view our contributions as laying the groundwork for efficiently computing Shapley variance and demonstrating how understanding Shapley variance is useful (e.g., characterizing when paired sampling helps Shapley value estimation).

Reviewer_GrysOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper provides an interesting perspective on SV estimation by focusing on variance estimation. Overall the presentation is clear and the scientific contribution is nontrivial. However I feel that some aspects deserve further clarification and revision before the paper can be considered for acceptance. See the points below.

Critical strengths

The research question is a real research gap; the presentation is clear and the results are insightful.

Critical weaknesses

CW1. The proposed estimator can be biased, and further analysis is needed.

The experiments report MSE but do not investigate the bias in detail. The proposed estimator could be biased because f^\hat{f} is fitted using the same coalition samples that are used to estimate the covariance and residual variance. Therefore it would be natural to expect a more detailed analysis of the estimator's bias and variance. Could the authors consider adding further results on the estimator's bias–variance tradeoff?

Minor Concern

While the paper has already provided some explanations of why Shapley variance estimation is useful, which I agree with, the authors may consider adding further illustrations using specific real-world applications to demonstrate when and why estimating Shapley variance is particularly important.

Questions for authors

See above. I'd raise my score if CW1 could be further investigated and addressed.

2 comments

We thank the reviewer for their time and thoughtful feedback. We appreciate their assessment that “the scientific contribution is nontrivial” and that “the results are insightful”.

CW 1

The experiments report MSE but do not investigate the bias in detail. [...] Could the authors consider adding further results on the estimator's bias–variance tradeoff?

To answer your question, we have run new experiments to measure the bias and variance of the estimators. For a fixed game, let v\mathbf v be its Shapley-variance vector and v^\hat{\mathbf v} an estimate. The mean squared error (MSE)--which we report in the paper--separates into variance and squared bias, with expectations over independent runs of the estimator:

\rVert_2^2 =\underbrace{\mathbb{E}\lVert\hat{\ \mathbf{v}}-\mathbb{E}\hat{\mathbf{v}}\ \rVert_2^2}_{\text{variance}} +\underbrace{\lVert\mathbb{E}\hat{\ \mathbf{v}}-\mathbf{v}\rVert_2^2}_{\ \text{squared bias}}.$$ The Monte Carlo estimator has a squared bias of zero because it is correct in expectation. In contrast, the estimators we propose reuse samples: once to fit the surrogate and once to estimate the residual error of the surrogate. Reusing the samples avoids splitting the query budget between these two steps, but can introduce bias. We compute both terms for our Shapley-variance estimates using 32 independent runs per estimator on the games and full FFD budgets of Table 1. The table below reports estimated squared bias as a percentage of MSE for each application, rounded to whole percentages. **For our proposed estimators, squared bias accounts for 0–3% of MSE in these application averages.** Variation across runs accounts for almost all of the measured error of our estimators. | Application | Polynomial | GP | Tree | FFD | MC | | --- | ---: | ---: | ---: | ---: | ---: | | Local explanations | 2% | 2% | 3% | 69% | 0% | | Feature retraining | 0% | 2% | 0% | 7% | 0% | | Training-group valuation | 1% | 2% | 0% | 87% | 0% | | Ensemble selection | 0% | 1% | 0% | 100% | 0% | | Transformer ablation | 1% | 2% | 3% | 50% | 0% | We have added Appendix F.2, which gives the full decomposition in Table 4 and Figure 9. ### Minor Concern > the authors may consider adding further illustrations using specific real-world applications to demonstrate when and why estimating Shapley variance is particularly important. We have expanded Figure 1 and added examples in Appendix A to show what Shapley variance reveals when players have similar Shapley values. We now illustrate Shapley variance in text, image, and tabular applications in Figure 1. (Thicker purple borders make the Shapley variance visible alongside the Shapley values.) In the new text example, we consider the sentence “The acting is good, but the plot is not great.” Here, “great” is a stronger positive word than “good,” but it is modified by “not.” The two words receive similar positive Shapley values while Shapley variance makes their different roles clear: “great” has much higher Shapley variance because its average contribution changes from positive without “not” to negative with it. Beyond the other examples in Figure 1, **Appendix A now has illustrations of Shapley variance for text, image, tabular, data valuation, and ensemble selection applications**. We now also describe two natural uses in the introduction: > There are many insights that Shapley variance can reveal in machine learning applications: > Low variance means a player's contribution is usually close to its Shapley value under the Shapley distribution. > Conversely, high variance identifies players whose contributions depend on other players. > Computing interactions among these players could focus the analysis on a smaller set of candidates, rather than evaluating every group up to a chosen interaction order (Tsai et al., 2023; Muschalik et al., 2024b).

The authors' revised manuscript has improved the clarity and confidence of this work; therefore, I am increasing my score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Changes

We thank the reviewers for their time and insights. To address their feedback, we have updated the paper in the following ways:

  • Illustrations and applications of Shapley variance: In response to reviewers Grys and Xg8a, we have redesigned Figure 1 to make the information from Shapley variance more clear. For example, the new text-classification example shows how the words "good" and "great" have similar Shapley values while Shapley variance reveals how "great" is modified by the qualification "not" before it. We have also added illustrations of Shapley variance in data valuation and ensemble selection applications in Appendix A.

  • The bias-variance tradeoff of our estimators: We use all samples both to fit the surrogate and to estimate its correction terms, which can introduce bias. In response to reviewer Grys, we have run additional experiments to quantify this bias: with 32 repetitions at the full FFD budgets in Table 1, our estimators' estimated squared bias accounts for 0–3% of MSE in the application averages.

  • Presentation: In order to address the concerns of reviewer 45Wn, we have:

    • Clarified our contribution as efficiently computing and estimating Shapley variance, building on existing methods for Shapley values.

    • Revised the related-work discussion to distinguish this computational contribution from the motivation for looking beyond average contributions.

    • Added a short explanation of surrogate-based Shapley value estimation in the preliminaries and clarified how we extend it to Shapley variance.

    • Summarized the proof techniques used in each result before stating the result.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.