Enforcing Conditional Independence in Flow Matching via Divergence-Difference Regularization

We study label-conditional fairness by encouraging a prediction score to be independent of sensitive attributes given the target label. We propose $\text{CI-CFM}$, a conditional flow-matching predictor with a compatible conditional-density regularizer. A normalized auxiliary model estimates group-conditional score densities, and their mixture defines the label-conditional reference. Evaluating both densities at the deployed score yields a differentiable log-ratio penalty, with density fitting and predictor updates alternated without adversarial training. The population construction recovers conditional mutual information when the component densities and mixing law match the true distributions; error bounds characterize departures from this identity. Across ten synthetic seeds, regularizing the deployed score rather than a single stochastic endpoint reduces mean residual HSIC by 38.7% without increasing mean prediction error. On Adult, CI-CFM lowers the mean equalized-odds gap from 0.1206 to 0.0628 and EDDI from 0.0663 to 0.0605, with small decreases in mean AUROC and F1. Across four MIMIC-III/IV settings, it achieves the highest AUROC and lowest EO and EDDI point estimates among the compared systems. CIFAR-10 and CelebA experiments retain comparable sample quality under an independent-nuisance null control. Code is anonymously available at https://anonymous.4open.science/r/CICFM-0B67/.