Conditionally Minimal Sufficient Representations for Debiased Visual Learning

Deep visual models often exploit spurious correlations in training data, degrading performance on bias-conflicting groups. Many representation-level debiasing methods reduce global dependence between representations and bias variables, which can be overly restrictive when the target and bias variables are statistically dependent. To address this limitation, we propose Conditionally Minimal Sufficient Debiasing Representation (CMDR), a conditional minimal-sufficiency framework that replaces global bias invariance with target-conditioned debiasing while limiting target-conditioned input redundancy. We theoretically characterize this formulation by establishing a feasibility separation between global invariance and conditional minimal sufficiency under a realizable setting, showing that $I(H;Z_b\mid Y)$ bounds target-conditioned group-risk gaps, and that $I(H;X\mid Y)$ bounds dependence on input-mediated nuisances under the corresponding Markov condition. CMDR instantiates these principles with a target prediction objective, a target-conditioned adversarial discriminator, and a lightweight variational regularizer, while supporting both explicit bias supervision and confidence-weighted pseudo-bias signals when training bias annotations are unavailable. Experiments on Waterbirds, CelebA, BAR, MetaShift, and BFFHQ demonstrate improved worst-group robustness across diverse visual biases and bias-conflicting ratios. Ablations show that target-conditioned debiasing outperforms global adversarial debiasing, while conditional minimality provides additional worst-group gains and confidence weighting improves robustness under weak supervision.