Adversarial Information Separation for Data-Scarce Generalization under Noisy Observations

Learning from scarce data is difficult, e.g., in scientific domains where task-relevant information is often confounded with measurement noise and experimental artifacts. Compression-based regularization like the variational information bottleneck (VIB) improves generalization by compressing representations uniformly, but may suppress weak predictive information along with noise. In contrast, adversarial separation methods preserve such information, but require explicit separation labels. To address this challenge, we propose the Adversarial Information Separation Framework (AdverISF), which learns primary and residual representations from task supervision alone. It uses a task-guided adversarial separation mechanism to reduce the statistical dependence between the representations and a multi-layer separation architecture in which subsequent layers process residual representations to extract predictive information missed by earlier layers. Experiments show that AdverISF outperforms state-of-the-art methods under severe data scarcity. On a real materials design task, it generalizes to formulations independently prepared in a subsequent batch.