acceptodds
Under review as a conference paper at ICLR 2027

GapTrace: Controlled Attribution of Prediction Disparities in Clinical Time Series

Abstract

Machine learning is increasingly used in clinical decision-making, raising concerns about both fairness and interpretability. These are usually studied separately: attribution methods explain which features influence predictions, while fairness analyses measure disparities without identifying the model behavior associated with them. We introduce GapTrace, a framework for auditing prediction disparities in time series models. Given a gap in predicted risk between patient groups, GapTrace measures how much of the gap remains after adjustment for correlated attributes, identifies features or feature clusters used differently across groups with false-discovery-rate control, and tests whether intervening on the selected features reduces the gap more than matched control interventions. GapTrace is agnostic to the attribution method and can be applied on top of existing tools. On synthetic and semi-synthetic datasets with known subgroup-specific mechanisms, GapTrace identifies the planted features (F1 up to 0.92, compared with 0.35 for random selection). On 30-day mortality prediction in MIMIC-IV, we find three different patterns. The largest prediction gap is across age groups (29.7 percentage points), with 92% remaining after adjustment for other demographics. GapTrace identifies 35 of 525 feature clusters with significant differences in attribution across age groups. Intervention on the selected features reduces the gap more than matched controls. In contrast, most of the marital-status gap disappears after adjustment, while the remaining insurance gap cannot be consistently attributed to a specific set of features. GapTrace turns a measured disparity into a testable explanation: which inputs the model relies on differently across groups, and whether they account for the gap, delivering fairness analysis and explainability together.

Primary area:
interpretability and explainable AI
Keywords:
algorithmic fairness · interpretability · feature attribution · clinical time series · subgroup disparities · counterfactual intervention · electronic health records . wearable data

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vrNdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes GapTrace, a framework for analyzing prediction disparities in clinical time-series models. It combines fairness measurement, feature attribution, statistical testing, and controlled interventions to identify features associated with prediction gaps across patient groups.

Critical strengths

The paper addresses an important problem in trustworthy machine learning. The motivation is clear, and the experimental evaluation is relatively comprehensive. I particularly appreciate the effort to connect fairness measurement, differential attribution, and intervention-based validation into a unified framework.

Critical weaknesses
  1. Novelty. The framework primarily combines existing techniques, including Integrated Gradients, DeepLIFT, statistical testing, feature clustering, and distributional interventions. While the integration is useful, I am not fully convinced that it constitutes a sufficiently novel methodological contribution. I would encourage the authors to further clarify what GapTrace offers beyond a straightforward combination of existing attribution and fairness auditing methods, ideally through additional comparisons or ablation studies.

  2. Explanatory power. In the MIMIC-IV experiments, the prediction gap between age groups is 29.7 percentage points, while the proposed intervention reduces it by only approximately 2.3 percentage points. This raises a question about how much of the observed disparity the framework can actually explain. Could the authors further investigate whether the remaining gap reflects limitations of the attribution methods or differences in model behavior that cannot be captured by the selected features?

  3. Intervention validity. The proposed interventions may disrupt dependencies among clinical features and generate unrealistic inputs. I encourage the authors to further discuss how this limitation affects the interpretation of the intervention results and whether more distribution-preserving interventions could strengthen the conclusions.

Questions for authors
  1. Novelty: What are the key methodological contributions of GapTrace beyond combining existing attribution, statistical testing, and fairness auditing techniques? Could the authors provide additional comparisons or ablation studies to better demonstrate its unique advantages?

  2. Explanatory Power: In the MIMIC-IV experiments, the intervention reduces the age-related prediction gap by only 2.3 out of 29.7 percentage points. How should we interpret the relatively limited reduction? Does it reflect limitations of the attribution methods, the selected features, or the intervention procedure itself?

  3. Intervention Validity: Since the proposed interventions may disrupt dependencies among clinical features, how can we ensure that the resulting gap reductions reflect meaningful model behavior rather than artifacts of unrealistic inputs? Could more distribution-preserving interventions help validate the findings?

2 comments

We thank the reviewer for the careful and constructive assessment.

  1. Novelty and methodological contributions: We agree that the individual techniques used by GapTrace, including Integrated Gradients, DeepLIFT, statistical testing, clustering, and distributional interventions, have established foundations. Our contribution lies in their organization into a disparity-auditing framework for multivariate clinical time series, together with a controlled evaluation protocol that tests recovery of known subgroup-specific mechanisms.

In particular, GapTrace connects statistically supported differences in model attribution, correlated and temporally structured feature groups, and targeted interventions evaluated against matched controls. These stages address distinct questions: whether model reliance differs across groups, which features exhibit these differences, and whether modifying those features changes the prediction gap. We have clarified these contributions and their relationship to existing fairness-attribution methods in the revised Introduction and Related Work sections. The revised evaluation also includes a FairCanary-inspired baseline (Table 1), against which GapTrace achieves higher feature-recovery F1 (0.92 vs. 0.79) and balanced accuracy (0.94 vs. 0.84) on the controlled synthetic benchmark.

  1. Explanatory power of the MIMIC-IV intervention: we agree that the approximately 2.3-percentage-point reduction is not a complete explanation of the 29.7-percentage-point age-associated disparity. Under the evaluated intervention, the gap decreases by approximately 7.7% relative to its original magnitude; the remaining 92.3% is not removed by this particular repair. This limited reduction is itself informative, but it should not be interpreted as a causal decomposition. GapTrace is designed to measure how much of a gap the selected features account for, and to report it honestly when that share is small. The age gap is a case where the model’s reliance on these features differs significantly across groups (35 of 525 clusters), and removing those differences reduces the gap more than matched controls, but only by a limited amount. We will revise the abstract and contribution statement so they do not imply that GapTrace explains the full disparity: it tests whether and how much the identified features account for it.

Several factors may contribute to the remaining disparity. The intervention modifies only selected features, while the model may retain age-associated information through other variables, correlated measurements, or interactions. Furthermore, marginal distributional alignment does not necessarily remove differences in joint dependencies or temporal relationships. Attribution-based selection is also sensitive to the attribution method and reference distribution.

To further investigate the scope of the intervention, we added an outcome-conditional performance-gap decomposition (Appendix A.9.2, Tables 13--14). Among survivors, patients aged ≥65\geq 65 exhibit a significant binary-cross-entropy error gap of 0.7210.721 (p=0.001p=0.001) relative to younger patients. Approximately 95%95\% of this gap is attributed to static diagnosis codes (0.6830.683), compared with 5%5\% to time-series inputs (0.0380.038). Five diagnosis codes show significant attribution differences after FDR correction, with the five largest contributions accounting for approximately 65%65\% of the error gap. Since the evaluated intervention modifies selected time-series features rather than static diagnosis codes, these findings provide additional context for its measured effect. Importantly, this decomposition concerns outcome-conditional prediction error, not the original 29.729.7-percentage-point prediction gap.

The extended analysis provides a more comprehensive characterization of age-associated disparities by examining contributions from both static diagnosis codes and time-series features. Together, these complementary analyses provide a broader understanding of the observed disparities.

  1. Intervention validity: We acknowledge that modifying clinical features may disrupt physiological and temporal dependencies, potentially producing out-of-distribution inputs. GapTrace's interventions are designed as diagnostic probes of a frozen model, rather than causal counterfactuals or clinically realizable interventions. To improve their interpretability, we employ three complementary strategies:
  • Rank-preserving distributional repair: Aligns group-wise feature distributions while preserving within-group ranks, although multivariate dependencies may change.
  • Joint feature transplantation (I2, Appendix A.2.7): Transfers selected features jointly from observed donor samples, preserving dependencies within the transplanted subset.
  • Matched-control interventions: Compare targeted modifications against the same perturbation applied to other model-relevant features selected for comparable attribution magnitude.

These strategies help assess feature-specific sensitivity, although none guarantees that modified inputs remain on the clinical data manifold. Accordingly, intervention results should be interpreted as model sensitivity under specified perturbations, not causal effects. The observed 2.3-percentage-point reduction exceeds matched random repairs (p = 0.048), but does not eliminate concerns about intervention realism.

We have strengthened the revised manuscript with an outcome-conditional performance-gap analysis (Appendix A.9.2), separating the contributions of static diagnosis codes and time-series inputs. We have also clarified that GapTrace's interventions are diagnostic probes of model sensitivity rather than causal counterfactuals. More clinically constrained interventions, including conditional sampling and matched trajectory transplantation, remain important directions for future work.

Reviewer_8yZ6OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors introduce GapTrace to audit prediction disparities in time series models. It measures subgroup risk differences, characterizes demographic adjustment, selects features or attribution clusters with group differences.

Critical strengths
  • Trying to identify what features cause the model disparity
  • Did the experiments in both sythetic and real-world data
Critical weaknesses
  • Not clear about the contribution of the work: the authros claimed that fairness and interpretability were evaluated seperately. But existing work such as [1-3] have done similar work. What is your novelty here?
  • The focus on 'time-series' is not clear to me. Seems the authors just choose it one case study, and the impact of 'time' here is not clear to me.
  • The results are mainly the variations of their proposed GapTrace vs random or matched control baselines. We do not know if the proposed work improves upon other baselines (e.g., differential-attribution method).

[1] "How Biased are Your Features?": Computing Fairness Influence Functions with Global Sensitivity Analysis [2] Explainability for fair machine learning [3] FairCanary: Rapid Continuous Explainable Fairness

Questions for authors

What is the novelty of the work compared with existing literature?

1 comment

We thank the reviewer for their comments. These points have helped us clarify the scope and contributions of GapTrace.

  1. Novelty and methodological contributions: We agree that fairness attribution and explainability have been studied in prior work, including the papers cited by the reviewer. GapTrace does not propose a new attribution estimator; it is designed to work with different attribution methods. Its contribution is an integrated, testable audit procedure tailored to subgroup prediction disparities in multivariate time-series models. Our contribution is an audit procedure for subgroup prediction disparities in multivariate time-series models, built so that its conclusions can be checked:
  • Statistically controlled localization. Differences in attribution distributions across groups are tested with false-discovery-rate control, not read off a descriptive ranking.
  • Correlation- and time-aware testing units. Correlated channels with similar temporal attribution patterns are tested jointly, which can improve sensitivity when signals are spread across redundant measurements.
  • Intervention against matched controls. We test whether removing group differences in the selected features reduces the gap more than the same transformation applied to other model-relevant features, with the model held fixed.
  • A benchmark with known ground truth. The planted group-specific mechanisms let us evaluate disparity localization quantitatively. This is not possible on observational data, where the true disparity-driving features are unknown.

We will revise the Introduction and Related Work to set these apart more clearly from FairXplainer, Fairness Shapley, and FairCanary.

  1. Why time-series structure matters: The most direct evidence is the time-resolved analysis in Appendix A.9.1 (Figure 8 - previous Figure 7). For MIMIC-IV mortality prediction (n = 3,822 stays, five age groups), GapTrace keeps attributions at each timestep and tests, for each feature and hour of the 24-hour window, how much of the attribution variance is explained by age group, with family-wise error thresholds. Two findings are obscured by time-aggregated importance:
  • Age effects are localized in time. Age-related attribution differences for chest-tube output, arterial blood gases, respiratory rate, Foley output, and blood pressure are concentrated in specific parts of the admission window rather than spread evenly over all 24 hours. Two features with the same aggregate attribution can therefore differ in when the model relies on them differently across age groups.
  • Effects are not monotonic in age. Comparing the between-group variance (η²) with Spearman rank correlations shows that several features have age gradients that hold only over restricted intervals, or that change sign. A single time-aggregated score per feature would average these patterns out.

Temporal structure also enters how units are built (§3.2, Eq. 4). Clusters combine cross-patient attribution co-credit with descriptors of each feature’s attribution trajectory: shape, temporal centroid, entropy, sign consistency, and autocorrelation. Features are grouped when the model relies on them on the same patients and at similar phases of the stay.

To be precise about scope: the primary FDR-controlled selection operates on time-aggregated feature or cluster attributions. The time-resolved analysis then localizes when the selected differences occur. We will state this division explicitly. This also sets GapTrace apart from single-prediction temporal explainers, which explain when one prediction depends on its inputs, not how temporal reliance differs between patient populations.

  1. Comparison with existing baselines: We agree that comparison with random selection alone does not show an advantage over existing differential-attribution methods. FairCanary provides a relevant attribution-based comparator for subgroup prediction differences. We have therefore extended our synthetic benchmark to include a FairCanary-inspired baseline. FairCanary uses Quantile Demographic Disparity Attribution (QDDA) to decompose differences in prediction distributions into feature-level contributions within prediction quantiles.

Because FairCanary produces continuous feature contributions rather than binary planted-feature predictions, we apply a permutation-based selection procedure to its resulting scores (the same selection procedure used for GapTrace to make the recovery comparison more consistent). This is an adaptation for our recovery benchmark, not a component claimed by the original FairCanary method. Across 50 synthetic runs with two planted disparity-related features:

MethodF1 ↑Balanced Accuracy ↑
GapTrace–Integrated Gradients0.92 ± 0.060.94 ± 0.04
FairCanary-inspired (B10, permutation)0.79 ± 0.050.84 ± 0.03

These results support improved recovery in this synthetic setting.

Reviewer_VnzoOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GapTrace, a framework for examining prediction differences between demographic groups in time-series models. It combines covariate adjustment, comparisons of feature attributions, and targeted interventions to investigate which inputs are associated with an observed prediction gap. The framework is evaluated on controlled datasets and a clinical mortality prediction task.

Critical strengths

I find the motivation useful. A group-level prediction gap alone tells us little about how a model produces that gap, and the paper tries to connect this observation to specific model inputs. I particularly appreciate the distinction between a feature being important for prediction and being relevant to differences between groups. The clinical analysis also allows for different outcomes: a gap may attenuate after adjustment, admit a feature-level explanation, or remain without a stable explanation. This makes the framework more useful than an analysis that assumes every observed gap must have a clear feature-level source.

Critical weaknesses

My main concern is that the paper leaves the meaning of “explaining a prediction disparity” somewhat unresolved. A feature can receive different attributions across groups because the groups have different input distributions, even when the model applies the same relationship to everyone. Alternatively, the model may rely on that feature differently depending on the surrounding inputs. These are different explanations, but the framework does not make sufficiently clear how its conclusions distinguish them. The paper acknowledges that its findings are associational rather than causal. My concern is more specific: even within an associational interpretation, what does the analysis establish about model behavior beyond identifying differences in attribution distributions?

This also makes the methodological contribution difficult to assess. Covariate adjustment, attribution comparison, and intervention each answer a useful question, but their answers do not automatically form a single explanation. Adjustment asks how much of a gap remains after accounting for other attributes, while intervention asks how predictions respond to modified inputs. I would like a clearer account of how these stages constrain one another and what additional insight their combination supports. As presented, I see a useful audit workflow, but I am less convinced that the paper establishes a sufficiently distinct explanatory contribution.

The temporal component raises a related question. Grouping features by their attribution trajectories may be useful for handling redundancy, but the paper could explain more clearly what this reveals about when or how a disparity develops. At present, temporal structure appears more clearly motivated as a way to organize the analysis than as a source of new understanding about the disparity itself.

Questions for authors

What precisely should a reader conclude when GapTrace identifies a feature set and intervening on it reduces the prediction gap? In particular, how should that conclusion distinguish differences in the populations’ inputs from differences in the model’s use of those inputs? It would also help to explain how the adjustment, attribution, and intervention stages jointly support this conclusion, and what the temporal analysis adds to the explanation beyond grouping related features.

1 comment

We thank the reviewer for the thoughtful and constructive assessment, especially for distinguishing differences in population inputs from differences in model use. We agree that the interpretation of each audit stage and their combined conclusions should be made more precise.

  1. Reader conclusion: The reviewer is right that a difference in attribution alone cannot separate the two mechanisms. Differential attribution identifies where group differences enter the model’s computation, not why. When GapTrace selects a feature set S and intervening on S reduces the gap beyond matched controls, the conclusion is this: the frozen model’s prediction gap is sensitive to group differences in S, more than to comparable differences in other features the model relies on to a similar degree. Which mechanism is at work is addressed by the intervention ladder (§3.4, App. A.2.7), not by attribution alone:
  • Marginal input differences: If rank-preserving marginal alignment reduces the gap, predictions are sensitive to marginal differences in (S), but this does not establish identical model use across groups.
  • Joint or contextual structure: If marginal alignment has little effect but joint transplantation (I2) reduces the gap, joint dependencies or contextual interactions may contribute. However, this does not uniquely establish context-dependent model use.

Our controlled experiments examine these mechanisms separately. In the synthetic benchmark, groups share input distributions but differ in the signals determining labels; GapTrace recovers the planted channels (Table 1). Conversely, injection design (i) (App. A.8, Table 5) introduces a measurement offset in an already-used feature. These experiments support the framework's ability to detect distinct sources of disparity, although planted label mechanisms do not alone establish the trained model's learned behavior.

  1. How the adjustment, attribution, and intervention stages constrain one another. The three stages provide complementary perspectives on prediction disparities, each addressing a distinct but connected question. Adjustment determines whether a disparity persists after accounting for demographic factors, attribution identifies features associated with the remaining gap, and intervention evaluates whether modifying those features meaningfully changes it. Together, these stages provide a more comprehensive and statistically supported assessment of prediction disparities than any stage alone, while maintaining an associational and intervention-specific interpretation.
  • Adjustment characterizes whether a disparity persists after conditioning on other demographic attributes and informs the interpretation and prioritization of subsequent feature-level analysis. If a gap largely disappears after conditioning on other demographic attributes, it is mostly inherited, and we do not search for features specific to it. The adjustment set excludes potential mediators (§3.1), so it cannot remove the proxy pathways the attribution stage is meant to find.
  • Attribution proposes candidates under error control. It narrows the search to units with statistically supported differential attribution (FDR-controlled). It also supplies the attribution magnitudes used to match controls, without which the intervention comparison would not be calibrated.
  • Intervention tests whether the candidates matter for the gap. Units can be attributed differently without moving the gap. The intervention evaluates whether modifying the selected units changes the gap more than comparable modifications to matched features.

We have also extended the MIMIC-IV analysis with an outcome-conditional performance-gap decomposition (App. A.9.2, Tables 13-14). This analysis distinguishes prediction differences among patients with the same observed outcome and separates the contributions of time-series inputs from static diagnosis codes. Among survivors, patients aged ≥65 exhibit a significantly higher binary-cross-entropy error than younger patients (gap 0.721, p = 0.001), whereas the difference among patients who died is not significant (gap 0.057). These results illustrate how complementary analyses can distinguish different aspects of an observed disparity without requiring a single causal explanation.

The added value of combining the stages is therefore not a single explanation. It is the ability to say, for each gap, which of these outcomes holds, and to avoid declaring a feature-level explanation from attribution differences alone. We will reframe the contribution in these terms.

  1. Temporal structure. We agree that temporal structure informs clustering (Eq. 4), while selection uses time-aggregated attributions. However, the time-resolved analysis (App. A.9.1, Figure 8), reveals when group differences in model reliance occur. In MIMIC-IV, age-related attribution differences are concentrated in particular intervals of the 24-hour window. We have clarified this distinction in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.