- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the limitations of completeness in additive attribution models. It shows that attribution rankings depend on the choice of downstream head and reference, even when completeness is satisfied. The authors use an additive time-series model and a Neural Additive Model to examine these limitations through theoretical analysis and empirical evaluation.
Critical strengths
- The paper includes extensive experiments across synthetic and real-world datasets. These experiments provide useful observations about the behavior of additive attribution.
Critical weaknesses
-
The main observation that completeness is insufficient to guarantee faithful attribution has already been extensively studied, including even theoretical results such as Bilodeau et al. (PNAS’2024). The paper mainly examines this known limitation in a specific additive setting, without establishing a sufficiently novel theoretical insight.
-
The core theoretical analysis is restricted to additive latent representations with linear downstream heads. This is a relatively narrow setting compared with general attribution methods, which can explain nonlinear models at dffierent modules or input level. It is unclear how the findings extend to broader attribution settings.
-
The paper covers many loosely connected findings, such as attribution limitations, downstream performance, evaluation, etc. The extensive discussion and secondary observations makes the main contribution difficult to follow. A more focused presentation would improve clarity.
Questions for authors
-
Given that the insufficiency of completeness is already well established, what fundamentally new theoretical insight does this paper provide beyond existing works?
-
What generalizable approaches for improving attribution reliability can be derived from these findings, particularly for attribution methods beyond additive representations with linear heads?