- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Temporal Distributional Values, which treat groups of input history as players and record paired changes in forecast quantiles or sampled trajectories as each group is added under shared randomness, yielding contribution distributions that reconstruct the full forecast. The authors validate reconstruction accuracy, mean consistency and computational efficiency on four models and three glucose cohorts, and provide case studies.
Critical strengths
This paper systematically extends distributional Shapley values to probabilistic time-series forecasting. It clearly distinguishes the explanatory scope supported by marginal quantiles and by joint trajectories, and it retains paired contributions under the same input order and shared randomness, so the baseline plus contributions exactly reconstructs the forecast distribution while the mean agrees with scalar Shapley values. Experiments cover four model types and three glucose cohorts, the numerical checks are thorough, the limitations are discussed candidly, and the time-in-range case studies have a clear application motivation.
Critical weaknesses
The core construction closely follows the Distributional Values of Franceschi et al. 2024. Reconstruction and mean recovery follow directly from a telescoping sum of successive differences, and most propositions restate existing results in the forecasting setting. We suggest clarifying the new theoretical contributions beyond that framework. The spread of the contribution distributions mixes three sources, input orders, reference histories and forecast draws, and variation across orders does not reflect predictive uncertainty. Although the appendix mentions this, the main results do not separate them. We suggest quantifying the variance from each source separately.
Questions for authors
Please refer to weaknesses.
4 comments
After reading Reviewer jZQ4's comments, I noticed additional weaknesses in the paper. In particular, the contribution distributions depend on the chosen coupling, meaning different couplings can produce different explanations while preserving the same marginals. Without a principled way to justify a specific coupling or bound this sensitivity, the resulting explanations risk being arbitrary and unreliable. Consequently, I am lowering my score to 2 to better align with jZQ4' evaluations.
Information about the choice of coupling
Thank you for raising this. We agree that two distinct pairing choices can preserve every forecast and still give different distributions of contributions. You ask whether this makes the explanations “arbitrary and unreliable.” However, the revised paper gives both a reason to choose a particular pairing and an exact account of what follows if one declines to choose it.
What is the theoretical reason for this choice of coupling? Franceschi et al. derive their Bernoulli Distributional Value using one uniform draw across coalitions. Their null-player example explains the choice. When adding a block leaves event risk at 50%, the shared draw gives zero switches, whereas independent draws disagree half the time. We use their construction and ask how far their switching score can vary across all pairings consistent with the forecasts. Franceschi et al., ICML 2024, §3.1 and Remark 3.6
Suppose adding a history block raises forecast event risk from 20% to 30%. Every pairing must switch at least 10% of outcomes. Applying both probabilities to the same uniform draw switches exactly 10%, all from absence to occurrence. Independent draws would switch 38%, despite the same ten-point change in risk. The shared threshold therefore counts only switches required by the change in risk, and it achieves that minimum simultaneously for every addition of a block. Matching percentiles at a single forecast horizon answers a similarly explicit question about how a given forecast percentile moves.
For a given history, reference and event, Proposition 11 and Corollaries 2–4 give the full attainable interval for each block’s switching probability. Every combination of values in those intervals is attainable under one coherent pairing across all input subsets. Consequently, block A switches the event more often than B under every pairing exactly when A’s lower bound exceeds B’s upper bound. This certified 750 of 2,250 comparisons in the strong-ranking controls (Appendix F.1). The temporal analysis also shows when to withhold a ranking. Requiring each forecast to preserve its next-step law given both observed pasts narrowed 576 of 768 pairwise intervals in four synthetic cases, yet the resulting bounds certified no strict block ranking there (Appendix B.2).
How does the coupling choice influence detection of faults? In our learned fault-localization study, the shared-threshold explanation can be computed directly from the forecast probabilities. For each block, average its risk increases and its risk decreases separately across input orders. These two quantities depend only on the coalition risks. Corollary 3 proves that any extra switching introduced by another pairing adds equal mass to both signs. Retaining the two quantities localized faults in 62.67% of 1,200 synthetic device histories, versus 52.53% for the validation-selected policy using signed scalar summaries (paired gain 10.14 percentage points, 95% interval 8.86–11.33). Computing them directly reproduced the same inspection decisions (Section 5.8 and Appendix F.4).
We believe this merits publication because it gives a forecaster a way to determine which conclusions require a pairing assumption and to test whether the information that survives can improve a decision. The revised paper supplies an exact test for the first in the binary-event setting and an independently graded example of the second. We respectfully ask you to reconsider the rating in light of those results.
The authors' have addressed some of my concerns, and thus I decide to raise my score.
Response to weaknesses
Thank you for your comments. In order to distinguish the added theory and separate the sources of explanation spread, we have revised the contribution statement and added the quantitative decomposition below.
Comment 1: "We suggest clarifying the new theoretical contributions beyond that framework."
Response: The added result concerns what a forecasting explanation can conclude when the pairing of forecast outcomes is unspecified. For a binary event, we determine the full range of possible contribution distributions for each input. We also prove that every combination of those individual distributions can be realized together by one consistent pairing construction (Corollary 2).
This gives an exact ranking rule. Define an input's switching probability as how often adding it changes whether the event occurs. One input has a larger switching probability under every allowed pairing if and only if its smallest possible value exceeds the other's largest (Corollary 4). The result distinguishes a ranking supported by the forecasts from one that depends on the selected pairing. We further separate changes in event probability from additional spread introduced by pairing (Corollary 3). These results are new to the theoretical basis of DV and specific to time series data.
A supporting temporal analysis computes tighter bounds when early forecast outcomes cannot be paired using later information (Appendix B.2). We credit the established probability and sequential-transport tools used. DV supplies the attribution operator, mean-recovery and efficiency results; the added contribution is this identification and interpretation analysis for forecasting. The characterization concerns each input's contribution distribution, rather than every possible joint dependence among inputs.
Comment 2: "The spread of the contribution distributions mixes three sources, input orders, reference histories and forecast draws, and variation across orders does not reflect predictive uncertainty. Although the appendix mentions this, the main results do not separate them. We suggest quantifying the variance from each source separately."
Response: We now quantify all three sources and their interactions. Intuitively, an explanation can change because we use a different reference history, reveal the inputs in a different order, or draw different forecast outcomes. Their combinations can matter too. These are sources of variation in the explanation, not interchangeable measures of uncertainty about the future.
For the duration target in our controlled dependence-stress setting with sampled empirical references, the shares of contribution variance are:
Source: Share Input order: 1.27% Reference history: 11.17% Forecast draw: 25.99% Order × reference: 2.73% Order × forecast: 7.11% Reference × forecast: 31.47% All three together: 20.26%
The interaction terms account for 61.57%, so reporting only three separate main effects would leave out substantial variation. We decompose each history/input combination first and then aggregate the components. The primary calculation uses 20 fixed histories, all six input groups, exact averaging over orders, 16 references and 256 forecast draws. With a fixed reference, all reference-related components are zero.
We also repeated the calculation across the full roster with 64 references and 1,024 forecast draws; substantial interactions persist. The added explanation in Section 4.1 points to these analyses, with numerical results in Table 16 and the complete decomposition in Appendices B.3 and E. Both budgets and their sensitivity are reported. These measurements concern the controlled synthetic study; sampling error is distinguished from contribution spread.
We have uploaded a revised manuscript and a supplementary version with the changes highlighted in yellow.