- Rating
- 4
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper gives methods to compute or estimate the variance of Shapley values. The importance in my own words and understanding is that the Shapley value is an average over coalitions and does not reflect if the attribute's contribution very strongly depends on, or interacts with, other attributes.
Exact algorithms are given to three classes of games, sparse polynomials (Fourier-like), trees, and Gaussian processes. For the general case, heuristics are given by using the above three possibilities as surrogates. The theoretical complexity is also analyzed and shown to be intractable.
I was searching for the literature for good illustrative examples on the use of the variance similar to that of the Shapley value but did not find good examples, which may partly due because prior to this result, no good methods were available to compute the variance. I like the examples provided in Appendix 1.
Critical strengths
S1 Providing efficient computational, exact or heuristic surrogate based, methods for computing the variance of Shapley values, a computationally intractable task. Experimentally showing the efficiency and accuracy compared to existing methods. Source code is available.
S2 Using the variance, giving theoretical insight into why paired sampling heuristic for Shapley values may work.
S3 Good illustrative examples
Critical weaknesses
W1 Limited motivation. The concept, although have already appeared in the literature, was less motivated by examples of usage. Beyond the general understanding that it is good to know if an attribute's contribution widely varies across coalitions, what is the insight? Should one separate attributes into groups in this case, build additive models, for example?
W2 Not independently of W1, I could not understand the examples in Fig 1, see Questions.
Questions for authors
Can you give illustrative examples on how to use the variance of the Shapley value for model explanations in the same way as motivated for the value itself? See W1.
Can you extend Fig 1? For the Housing example, I see no difference in the Shapley value and the Shapley + Variance charts, values on bar labels are the same, what does it mean? For the cat example, what does it mean that Patch B interacts with other patches, can you give visual examples?
2 comments
Maintaining my support of acceptance
I am satisfied with the answers to my as well as to other reviews. I maintain my already positive recommendation.
We thank the reviewer for their time and helpful suggestions. We appreciate their recognition of our methods' “efficiency and accuracy” and the “theoretical insight” our analysis gives into paired sampling.
Shapley variance can help us decide how much to trust an average contribution and where to look for interactions.
Low variance means the Shapley value represents the player's contribution across most coalitions under the Shapley distribution. This is particularly useful for a player with a near-zero Shapley value: low variance means its contribution is usually small, whereas high variance means the average can conceal substantial positive and negative contributions.
High variance gives us a reason to investigate interactions involving that player. We now lead Figure 1 with a text example: “The acting is good, but the plot is not great.” Both “good” and “great” receive similar Shapley values, even though “great” is a stronger positive word modified by “not.” Its higher variance makes this difference visible and motivates examining how its contribution changes when “not” is present.
We added these uses to the introduction. Evaluating whether variance can guide more efficient interaction analysis is a natural next step that our computation methods make possible.
We have revised Figure 1 to make it more clear: The first column shows the Shapley value explanations while the second column shows the same explanations with Shapley variance highlights. In particular, the thickness of the purple border around each player indicates the magnitude of its Shapley variance.
In the housing example, rooms and bedrooms have similar Shapley values, but bedrooms contribute positively on average without floor area and negatively with it. In contrast, rooms contribute negatively in every coalition. The thicker border around bedrooms shows their higher Shapley variance, revealing a difference that the similar Shapley values conceal.
We meant that a region's contribution depends on which other regions are visible, but the cat example did not show this clearly. We have replaced it with a sea-slug image: Body region A and backdrop region B have similar Shapley values, but A has much higher Shapley variance. A contributes more on average when adjacent body region C is visible, while B changes much less.