- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper develops a model that predicts future health measurements from past records and makes the attribution of feature sets explicit. It aggregates each feature’s values and combines these aggregates using a learned score for every possible feature set. The paper’s main technical proposal is a recursive rule that keeps these scores consistent with the requirement that adding a feature cannot decrease the set’s score. Experiments evaluate interaction recovery on synthetic data, as well as prediction accuracy and explanation stability on clinical health records. The proposed model recovers interactions well when the synthetic data follow its assumed structure and gives more repeatable pairwise and three-way interaction explanations on clinical data, but requires many parameters and predicts less accurately than boosted trees.
Critical strengths
The strengths of this work are not substantial: (1) the model is interpretable, i.e. it shows how individual features and combinations of features contribute to its predictions, and (2) the paper evaluates explanations on synthetic data while also providing a real-world application in the medical domain.
Critical weaknesses
Two critical weaknesses of this paper are (1) lack of novelty/significance and (2) poor quality in writing.
- Firstly, the paper does not clearly explain why the research matters, and the empirical results reported in Section 6 do not appear to constitute a substantial advance in machine learning. Furthermore, the theoretical novelty of Section 4 appears limited. The marginal contributions, Möbius coefficients, and Choquet decomposition are classical, while Theorem 1 follows from Möbius inversion and elementary counting. Theorem 2 combines the standard indicator-input property of the Choquet integral with straightforward error bounds; it does not establish recovery from observational longitudinal data. These results help explain the model, but neither theorem depends on the proposed recursive parameterization or longitudinal architecture. Finally, the paper does not engage well with related literature on higher-order feature interactions; only for example:
[a] Detecting Statistical Interactions from Neural Network Weights. ICLR 2018
This work identifies higher-order interactions from trained network weights and should be discussed alongside the paper’s synthetic interaction-recovery experiments.
[b] Neural Interaction Transparency (NIT): Disentangling Learned Interactions for Improved Interpretability. NeurIPS 2018
NIT learns interpretable additive models with interactions of controlled order, offering a natural comparison for assessing the benefit of retaining higher-order terms.
[c] Exploring Interpretable LSTM Neural Networks over Multi-Variable Data. ICML 2019
IMV-LSTM already uses variable-specific temporal representations to support interpretable prediction, and the paper should explain how its feature-preserving longitudinal aggregation differs from this approach.
[d] How does This Interaction Affect Me? Interpretable Attribution for Feature Interactions. NeurIPS 2020
Archipelago separates interaction detection from attribution to individual predictions, an issue that also arises in the paper’s distinction between capacity interactions and explanations of forecasts.
[e] Scalable Interpretability via Polynomials. NeurIPS 2022
SPAM uses low-rank polynomials to model higher-order interactions efficiently, making it a particularly useful comparison given the exponential size of the complete fuzzy measure.
[f] HarsanyiNet: Computing Accurate Shapley Values in a Single Forward Propagation. ICML 2023
HarsanyiNet builds Harsanyi interactions into the predictor and derives exact Shapley values from them, so the discussion of explanations obtained directly from the model could acknowledge this earlier architecture.
[g] Learning to Understand: Identifying Interactions via the Möbius Transform. NeurIPS 2024
This work studies recovery of Möbius coefficients from function evaluations under sparsity assumptions and should be discussed in connection with the paper’s complete-probe identification result and its computational cost.
[h] InstaSHAP: Interpretable Additive Models Explain Shapley Values Instantly. ICLR 2025
InstaSHAP connects generalized additive models with Shapley explanations and trains interpretable models to compute these attributions in a single forward pass, directly relating to the paper’s integration of prediction and explanation.
- There are too many issues with the paper's clarity for it to be readable by the ICLR audience, e.g.:
- In the Abstract, the sentence "Across a harmonized set of clinical cohorts containing 166,836 participants, we predict 12 biomarkers and two depression scores with standardized test mean squared error 0.5215." is not informative: why is "harmonized" or the number of participants important? What are these biomarkers? This value of "0.5215" is not informative without a baseline comparison. Further, "we obtain the most stable pair and triple explanations among these predictors, with triple cosine similarity 0.677 across training seeds" the most stable compared to what? What is this cosine similarity measuring?
- The introduction has no references to the literature, which makes its contribution to machine learning research unclear.
- in Equation 1 is not explained.
- In "The main model uses softmax weights.", it is unclear what are the non-main models and how do "softmax" weights look like.
- Section 3 could start by stating what the goal of this method is, and revisiting "Why?" at each point of describing "what" and "how" it was done.
- Proposition 1 is not understandable; each of the terms "monotone," "capacity," and "parameters" should be better explained.
- L163 writes about "biomarkers" and "depression scores," but the reader does not know what they mean at this point.
- L168 "they are not changes in clinical outcome units" is unclear.
- Section 5 is too verbose, and much of it could have been stated in the Appendix, making space for actual results.
- There are too many numeric values appearing in the text of Section 6 without a clear explanation for why they matter, e.g. "MSE of 0.51492, compared with 0.52148" or "with cosine similarities from −0.036 to 0.066".
Questions for authors
I don't think answers to any question could affect my assessment for this work.
5 comments
2. Related literature
Thank you for the list. We checked each reference, and we now discuss all eight in Related work, with a comparison table in Appendix A.3 (Table 4). Where a method could be run in our protocol, we ran it (Appendix G).
- [a] We ran NID (ICLR 2018) on the signed synthetic generator. It ranks the planted pairs first to third and the planted triple second to fifth. It gives no signs, though, so the negative pair is ranked only by strength (Table 37).
- [b] We discuss NIT (NeurIPS 2018) as a neural model with controlled interaction order. We did not run NIT itself. The question it raises, whether keeping higher orders helps, is tested with k-additive capacities, EBM, and SPAM in the same protocol.
- [c] We discuss IMV-LSTM (ICML 2019): it keeps a separate state for each variable and reports variable importance, not set interactions. We also ran an IMV-LSTM-style model on the synthetic data. No official implementation exists for our longitudinal setting, so we label it as our own adaptation. It fits the generators poorly, so its results say little about the original method.
- [d] We discuss Archipelago (NeurIPS 2020) alongside our separation of capacity coefficients from explanations of final predictions.
- [e] We ran SPAM (NeurIPS 2022) at order 2 and 3 on both synthetic generators and on the clinical data. SPAM of order 2 predicts better than our model (0.5074 against 0.5215), but its pair explanations are much less stable across seeds (cosine 0.226 against 0.778). SPAM of order 3 also predicts better (0.5098), and its triple explanations are much less stable (cosine 0.047 against 0.677).
- [f] We agree that HarsanyiNet (ICML 2023) is the closest in spirit, and we now say so. The difference is that our coefficients form a normalized monotone capacity over named longitudinal summaries, with every set enumerated. We ran a HarsanyiNet-style model on the synthetic data, again labeled as our own adaptation. It also fits poorly, so we do not read the result as a verdict on HarsanyiNet itself.
- [g] We discuss Kang et al. (NeurIPS 2024) next to our complete-probe result. Sparse Mobius recovery from queries is a natural route past the cost of enumerating every set.
- [h] We discuss InstaSHAP (ICLR 2025) next to explanations obtained in one pass from interpretable additive models.
We also added iChIMP and EBM as direct comparisons.
3. Clarity
Thank you for the precise list. We addressed every item.
- We rewrote the abstract. It now names the cohorts (CARDIA and WHI) and the outcomes, and says what the cosine measures: agreement of signed triple attributions across training seeds. Every number now comes with its comparison (for example 0.68 against 0.05 to 0.45), and the abstract states the accuracy cost.
- The introduction now has references. It cites work on the clinical motivation, sequence models, interaction attribution methods, interpretable interaction models, and fuzzy measures.
- z_ij in Equation 1 is now defined as the scalar summary, between 0 and 1, of participant i's history of feature j up to the landmark; it is the Choquet input for feature j. The weights, elapsed time, and transform are also defined.
- For "The main model uses softmax weights", the softmax formula is now given. We also say that mean, sigmoid, and power-mean weights were the alternatives compared on validation data (Appendix E.2).
- Section 3 now states its goal and the "why". It opens with the goal and three requirements (identity, validity, and exactness), and points each requirement to the subsection that addresses it. A new paragraph explains why we summarize features separately, with a direct test against a joint encoder.
- For Proposition 1, "capacity" and "monotone" are now defined in Section 3.2, before the proposition. A plain-language reading follows it: any real-valued choice of the parameters gives a valid capacity, and every capacity with strict increases can be reached, so training needs no constraints.
- On biomarkers and depression scores, the introduction now states the clinical task, and Section 5.3 describes the 21 inputs (ten measured biomarkers and eleven diet components) and the 14 outcomes.
- We replaced "not changes in clinical outcome units" with an explicit statement of how a set term translates into outcome units through the decoder (end of Section 3.4).
- We condensed Section 5 to the goal, data, and metric of each experiment. Preparation details and hardware moved to Appendices D and E.5.
- For the numbers in Section 6, we removed five-decimal values from the text and tried to pair each remaining number with its comparison and its meaning, for example "about 3 percent higher than trees". The example you quote, cosine -0.036 to 0.066, is now explained as disagreement between predictors on triples, and it is used to make the point that stability is not correctness.
Thank you again for the detailed reading.
Thank you for the expert reading and for the specific references. Your comments led to many of the largest revisions. We agree with several of your points, and we changed the paper rather than argue them. The contribution we now claim is the combination of three things: longitudinal summaries that keep each feature's name, a headroom-scaled complete capacity, and a common evaluation of prediction explanations across model families. Section and table numbers refer to the revised PDF.
1. Novelty and significance
We now explain why the research matters. The introduction starts from a concrete problem, with references. Clinicians often need to know which combinations of risk factors drive a forecast. Sequence models only allow post hoc interaction attributions, and those can change when the model is retrained. Our own data quantify this: a Transformer's triple attributions agree across retraining seeds with a cosine similarity of only 0.10, and different predictors barely agree on triples at all (cosine -0.04 to 0.07). A model whose interactions are part of the prediction is one way to address this.
Here is what we claim now, and what we no longer claim:
On the theory, we agree with your assessment. The set-gain identities and the complete-probe result are classical consequences of Mobius inversion and the indicator property of the Choquet integral. We have:
The only formal result tied to our parameterization is Proposition 1, the validity and one-to-one property of the headroom-scaled recursion. We now support it with a direct comparison against the clipped iChIMP construction (Section 6.2). We also narrowed that claim. Headroom scaling avoids hard clipping and the dead increments of a ReLU; it does not give a larger class of capacities, and it does not guarantee that every parameter receives an informative gradient. The comparison supports an optimization benefit in the settings we tested. A softplus variant of iChIMP essentially matches our static recovery and clinical prediction, and we did not evaluate its clinical stability.
On recovery from observational data, you are right that Lemma 2 does not establish recovery from observational longitudinal data, and we now say so right after the lemma. To make the point concrete, we added an identifiability diagnostic on the clinical training data (Section 4.2, Appendix B.4, Tables 5 and 6). A set value receives direct evidence only if the set appears on some participant's sorted chain. The diagnostic uses the summaries of one representative fitted model.
The favorable conditioning therefore concerns those selected columns only. It does not establish identifiability of the full jointly learned model. So most high-order clinical set values are shaped by the recursion rather than by the data, and we list this as a limitation.