- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
Shapley values can measure the contribution of individual input features to a model's prediction. This paper proposes to separate these contributions into reliable and unreliable components using AND-OR interactions between features. The authors express each Shapley value as the sum of contributions from interactions involving the corresponding feature. They prove that this reconstruction has a bounded approximation error. They then classify an interaction as reliable if it appears in both the original model and a reference model and has the same positive or negative effect on the prediction.
Critical strengths
- The reconstruction result provides a mathematical basis for separating attributions according to interaction properties.
- The empirical evaluation is extensive in terms of language models.
Critical weaknesses
- The method first represents a model's predictions using AND-OR interactions. However, Appendix F.1 states that the optimization used to extract these interactions can have multiple optimal solutions. These solutions may produce different interactions while representing the same model predictions. Since reliability is assigned to individual interactions, the resulting reliable Shapley values may depend on which solution is selected. The paper does not establish whether these components remain stable across different optimal solutions. The theoretical error bound only controls the sum of the reliable and unreliable components.
- The experiments do not establish whether cross-model agreement is responsible for the observed improvements. The authors show that reliable components are more stable and assign more importance to relevant tokens. However, Figure 5 also shows that reliable and unreliable interactions differ in the number of features they involve. This difference may partly explain the results. Comparing interactions with similar sizes and contribution magnitudes would help determine whether agreement across models provides an additional benefit.
Questions for authors
- Appendix F.1 acknowledges that the interaction-extraction problem may have multiple optimal solutions. Can these solutions produce different reliable and unreliable Shapley components for the same model? Could you evaluate how much the components change across different optimal solutions, or establish conditions under which they remain unchanged?
- Figure 5 shows that reliable and unreliable interactions have different distributions across interaction orders. Could you compare two methods for selecting interactions? Method A uses your proposed criterion. It selects interactions that also appear in a reference model with the same contribution sign. Method B does not use a reference model. It selects the same number of interactions while matching their orders and contribution magnitudes to those selected by Method A. Could you then compare the resulting Shapley components using the metrics already reported in the paper? This would help determine whether agreement across models provides benefits beyond simply selecting interactions with these characteristics.