Causal-Driven Feature Evaluation for Cross-Domain Image Classification

Out-of-distribution (OOD) generalization remains a core challenge in real-world classification. Most existing methods for OOD and domain generalization focus on cross-domain stability. However, cross-domain stability alone does not imply causal effectiveness. A feature may appear stable across source domains, yet fail to truly drive classification decisions under distribution shift. We re-evaluate learned representations from the perspective of causal driving force. By intervening on feature representations, we explicitly measure the necessity and sufficiency of each representation segment for cross-domain prediction and convert these measurements into causal effectiveness scores. Based on these scores, we select high-scoring segments to form a representation subspace and retrain the classifier on this subspace, encouraging the model to rely more on representation components that drive classification and thereby improving robustness on unseen domains. We validate the proposed method on multiple datasets and pretrained backbones, where it consistently improves classification performance on unseen domains. We further provide theoretical derivations for the proposed method.