acceptodds
Under review as a conference paper at ICLR 2027

Unsupervised Domain Shift Detection with Interpretable Subspace Attribution

Abstract

We developed a tool for detecting domain shifts, namely subtle differences in the probability distributions of datasets. We identify these shifts using an algorithm designed to detect localised density differences in high-dimensional feature spaces. If an anomaly is present, we then identify the subset of features in which the anomaly is most pronounced. This allows to trace the domain shift to a small set of features, making the shift interpretable. Moreover, we provide a protocol for compensating domain shifts by extracting, from two unlabelled datasets, subsets of samples in which the anomaly is not present. We validate the framework on controlled 20-dimensional benchmarks with known ground truth, recovering both broad and localized shifts together with their supporting feature subspaces. We then apply it to healthy electrocardiogram (ECG) recordings represented by 782 features. In age- and sex-matched cohort comparisons differing in measurement-device composition, the method detects device-induced shifts, extracts representative subsets enriched in the imbalanced device components, and identifies ECG features associated with the anomaly.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Distribution Shift, Unsupervised Learning, Two-Sample Testing, Anomaly Detection, Interpretable Machine Learning, Subspace Learning

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 2 reviews, above 9% of the others.
Reviewer_waChOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a three-stage, label-free pipeline for detecting, localizing, and attributing covariate shift between two unlabelled datasets. It consists of a pointwise backbone (based on the EagleEye method), a tail-based equalization step, and a Neighbor-Enriched Subspace Localization method that learns non-negative per-feature attribution weights. Validation has two parts: a 20-dimensional Gaussian-mixture benchmark with known ground truth, run across multiple seeds with parameter-sensitivity grids, as well as an application to PTB-XL+ healthy ECG recordings (782 features), where age/sex-matched cohorts are constructed to differ in measurement-device composition. The method prunes 3–6% of each cohort, the pruned sets are enriched in the imbalanced device components, while selected supports span 120–140 of 782 features.

Critical strengths
  • The controlled benchmark demonstrates the capability of the model to detect localized shifts. The model is able to recover a large percentage of the injected samples in most experiments.

  • The ECG example is a well-designed real-world experiment for this problem.

Critical weaknesses
  • The pointwise anomaly scores are calibrated using B(i, K), which depends on a sample's neighbors, which is OK. But then, the tail scores are treated as an i.i.d. sample in a two-sample KS comparison, whereas the scores of nearby points are computed from largely the same pooled neighbours and are strongly dependent. Moreover, Algorithm 1 performs repeated tests without adjusting the significance level to control for the error introduced by multiple testing. As a result, the calibration of the stopping rule needs additional theoretical and empirical validation.

  • The comparative claims are not supported by the required experiments. The introduction motivates the work against MMD, energy distance, C2ST and nearest-neighbor two-sample tests, however none of them is evaluated quantitatively. Furthermore, there is no detection experiment (e.g., power/false-positive at matched level) for any method, including the proposed. And the attribution stage is never compared against any alternative, even though the true support is known exactly on the benchmark. Ablation studies measuring the contribution of each component (equalization alone vs plus NESL) are also not evaluated.

  • Positioning compared to the relevant literature is incomplete. There exist approaches in the literature that deal with the same problem. For example, Kulinski et al., "Feature Shift Detection: Localizing Which Features Have Shifted via Conditional Distribution Tests", NeurIPS 2020; Barrabés et al., "Adversarial Learning for Feature Shift Detection and Correction", NeurIPS 2023; and others. These methods are not discussed or compared in the paper.

Questions for authors
  • How were baseline hyperparameters determined? And why does it make sense to keep them constant? For example, does the MLP training converge after only 300 training iterations?

  • How does the proposed method relate other methods in the literature?

Reviewer_JiiVOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper extends EagleEye with bidirectional sample removal and feature-subspace learning to localize and explain distribution shifts. Synthetic experiments demonstrate improved sample localization and recovery of designated feature subsets. On PTB-XL+, the method identifies discrepancies associated with ECG device composition, while downstream classification results show mixed effects of sample removal.

Critical strengths

The paper addresses the useful goal of identifying both samples and features associated with distribution shifts.

Critical weaknesses

NESL’s neighborhood-enrichment objective is closely related to Neighborhood Components Analysis, using dataset membership as labels and restricting learning to detected anomalous samples. The paper neither adequately positions this connection nor compares against an appropriately adapted NCA baseline. Existing comparisons primarily establish synthetic sample-localization gains, leaving the feature-attribution contribution insufficiently validated. The PTB-XL+ results demonstrate device-associated discrepancies, but mixed downstream outcomes on changing evaluation cohorts do not establish the practical benefit of equalization. The choice of PTB-XL+ is also questionable for the paper's purpose. Matching age and sex does not eliminate other sources of inter-subject variation, so the detected discrepancies may reflect physiological difference on the individual level.

Questions for authors

See the weaknesses above. Please clarify:

  1. What distinguishes NESL from an NCA formulation adapted to dataset membership and anomalous queries?
  2. Can the contributions of NESL and bidirectional equalization be evaluated separately?
  3. How can device effects be distinguished from patient-composition differences in the PTB-XL+ experiments?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.