- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MAICP, a model-agnostic framework for identifying invariant features across environments. It uses density adjustment and held-out prediction scores to reliably test conditional invariance across environments, even when the prediction model is misspecified. Experiments show that MAICP maintains reliable test calibration, identifies stable features in learned representations, and improves prediction accuracy under constructed distribution shifts.
Critical strengths
-
The paper provides a density-adjusted invariance test with asymptotic Type I error control, even under model misspecification. It addresses a limitation of existing invariant prediction methods and strengthens the reliability of invariant feature selection.
-
The paper demonstrates the practical applicability of MAICP to learned representations. Experiments with CLIP features show that MAICP can identify invariant semantic features and improve prediction accuracy under distribution shifts.
Critical weaknesses
-
The paper mainly evaluates linear and logistic regression models under relatively simple settings. Experiments with more complex prediction models and diverse data-generating mechanisms would provide stronger evidence for the general applicability of MAICP.
-
The paper does not sufficiently demonstrate the advantages of MAICP over existing methods. The baseline comparisons are limited and mainly presented in the appendix. Although the authors identify model misspecification as a key challenge, the experiments primarily compare MAICP with linear ICP and IRM. More direct comparisons with related methods, such as nonlinear ICP and conditional distribution testing methods, would help establish whether MAICP offers meaningful improvements over existing approaches.
Questions for authors
-
How does MAICP compare with existing nonlinear invariant prediction methods?
-
How well does MAICP perform with more complex models, such as DNN and transformers?