- Rating
- 4
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes OperatorSHAP-IQ to efficiently estimate spatial interaction maps for neural operators. The work extends Shapley attribution and interaction analysis from discrete features to continuous input fields, and further extends the Aumann–Shapley framework from first-order attribution to higher-order interactions. The proposed amortized explainer predicts interaction fields with a single forward pass after training. Experiments cover 12 PDE datasets across 1D, 2D, and 3D settings, including both regular grids and irregular meshes.
Critical strengths
-
I think the amortization idea is practically useful. If we need explanations for many samples, training an explainer once and then getting an interaction map with a single forward pass is much more convenient than repeatedly evaluating a large number of coalitions. The experiments are also fairly broad, covering 1D, 2D, and 3D problems, as well as regular grids and irregular meshes.
-
The paper is also theoretically solid. The connection from finite-player Shapley interactions to continuous interaction fields is developed carefully, and the proposed method follows this theory quite closely. I also like that the experiments include several sanity checks that are consistent with the theoretical expectations.
Critical weaknesses
The scientific value of the explanations is still not very clear to me. Most of the presented examples look more like sanity checks: the linear heat equation has almost no interaction, while the nonlinear Burgers equation shows clear interactions, which is largely expected. What is less clear is what we can actually do with these interaction maps once we obtain them. For example, if the learned interaction pattern is very different from what we expect from the underlying physics, can this information be used to diagnose or improve the neural operator?
Questions for authors
What do you think about the masking strategy for continuous spatial inputs? Hard masking a spatial region with zero or a mean value can introduce artificial discontinuities or OOD states that are not present in the original data. Since Shapley values are very sensitive to the masking baseline, how do you exclude the bias brought by different baseline.
2 comments
Thank you for the response, I will keep my positive evaluation.
Thank you for your very positive review. We are happy that you found our idea useful. As for your remaining questions:
R1: In general, the value of attributions and interactions lies in sanity checks for the user and in model diagnostics for the developer. As you correctly identified, if an explanation deviates from what we expect from the physics, a user could choose not to trust a prediction and a developer could investigate why their model would behave in such a way. However, amortized explainers are usually not necessary for research-scale problems. Our method aims more for commercial usage, where upcoming regulations could force companies to monitor their models behavior and per-sample predictions would be too costly.
R2: No, the input stays continuous, but the justification for this was only in the original OperatorSHAP paper and we should add it here as well. The reason is that models usually act on a discretized version of the data. For example, instead of getting a continuous input over the interval [0,1], the data is only present at locations {0,41,21,43,1}. That means, even if we use hard masks on that discrete locations, the models internally assume a smooth transition between the nodes, so no discontinuities are introduced.
R3: You are correct that this bias exists and we do not exclude it. The choice of baseline is always a parameter that the user has to decide, as it is in discrete Shapley. As we detail in Appendix B12 and ablate in Table 12, the choice of baseline has an effect on the interpretation of the explanations, but not on their quality. When providing an interpretation, the user always has to report the baseline they used and adjust their interpretation accordingly.
We have updated the manuscript with your suggestions. Thank you again for your recommendation.