acceptodds
Withdrawn from ICLR 2027

Enforcing Conditional Independence in Flow Matching via Divergence-Difference Regularization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We study label-conditional fairness by encouraging a prediction score to be independent of sensitive attributes given the target label. We propose , a conditional flow-matching predictor with a compatible conditional-density regularizer. A normalized auxiliary model estimates group-conditional score densities, and their mixture defines the label-conditional reference. Evaluating both densities at the deployed score yields a differentiable log-ratio penalty, with density fitting and predictor updates alternated without adversarial training. The population construction recovers conditional mutual information when the component densities and mixing law match the true distributions; error bounds characterize departures from this identity. Across ten synthetic seeds, regularizing the deployed score rather than a single stochastic endpoint reduces mean residual HSIC by 38.7% without increasing mean prediction error. On Adult, CI-CFM lowers the mean equalized-odds gap from 0.1206 to 0.0628 and EDDI from 0.0663 to 0.0605, with small decreases in mean AUROC and F1. Across four MIMIC-III/IV settings, it achieves the highest AUROC and lowest EO and EDDI point estimates among the compared systems. CIFAR-10 and CelebA experiments retain comparable sample quality under an independent-nuisance null control. Code is anonymously available at https://anonymous.4open.science/r/CICFM-0B67/.

Primary area:
generative models
Keywords:
conditional independence;flow matching;conditional mutual information;conditional density estimation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_zojROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CI-CFM, a fairness-aware prediction framework that encourages prediction scores to be independent of sensitive attributes given the true outcome (i.e., the equalized odds). The main contribution is a density-based regularizer that compares score distributions across sensitive groups without adversarial training. The authors also apply the regularizer directly to the final prediction score rather than individual stochastic outputs. They establish a theoretical connection to conditional mutual information and provide bounds on approximation errors.

The method is evaluated on synthetic regression data, Adult, MIMIC-III/IV, and image generation benchmarks. On synthetic data, it reduces conditional dependence compared with existing regularizers while maintaining similar prediction accuracy. On Adult, it substantially improves equalized odds with a small accuracy trade-off. The clinical experiments also report favorable fairness and predictive performance, while the image experiments show comparable generation quality.

Overall, the results are promising, although it remains unclear whether the proposed regularizer offers similar benefits beyond flow-matching models.

Critical strengths
  1. The proposed density-based regularizer avoids adversarial training and directly targets the prediction score used at inference. The theoretical connection to conditional mutual information provides a sound motivation for the method.

  2. The experiments show consistent reductions in conditional dependence on synthetic data and improved fairness on real-world datasets, with limited loss in predictive accuracy. The ablation studies also support the importance of regularizing the deployed score, strengthening the empirical evidence for the proposed design.

Critical weaknesses
  1. The discussion of closely related work and theoretical guarantees is somewhat insufficient: The paper does not adequately discuss classical existing work such as "Fairness-Aware Learning for Continuous Attributes and Treatments", which is closely related in its treatment of conditional independence using mutual information, fairness objectives, and theoretical guarantees. For the treatment of multiple sensitive attributes, papers such as "FairICP: Encouraging Equalized Odds via Inverse Conditional Permutation" address joint dependence across multiple sensitive attributes, whereas the proposed method averages separate penalties for each attribute, which does not necessarily guarantee joint independence. A more careful comparison is needed to clarify the novelty of the proposed regularizer, its advantages over existing adversarial and non-adversarial approaches, and what convergence or fairness guarantees it provides.

  2. Unclear fundamental novelty and motivation for flow matching: The density-based regularizer appears applicable to general differentiable predictors, and conditional independence is already a well-studied fairness objective (i.e., equalized odds). It is unclear whether the paper addresses a genuinely challenging new setting or introduces a fundamentally different solution to an existing problem. The authors should better justify the need for a flow-matching framework and demonstrate its advantages over simpler alternatives.

Questions for authors

See weaknesses.

Reviewer_SveHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CI-CFM to encourage conditional independence between prediction scores and sensitive attributes given the true label. It uses a density-ratio regularizer and applies it directly to the final prediction score used at inference. Experiments on synthetic data, Adult, MIMIC-III/IV and image datasets show improved fairness and conditional dependence metrics, while keeping similar predictive performance.

Critical strengths

1, Applying the regularizer to the actual prediction score instead of a single stochastic endpoint is a reasonable idea. The synthetic experiments show this design can make a difference

2, The evaluation includes several datasets and different fairness metrics. The results on clinical datasets are also useful for showing the potential applications.

Critical weaknesses

1, The theoretical novelty is quite limited. Using density ratios for conditional mutual information is already well studied. The main contribution seems to be applying this idea to flow matching rather than proposing a fundamentally new method

2, The method depends on estimating conditional densities accurately. Not clear how robust it is when some groups have very few samples, or when the test distribution differs from training.

Questions for authors
  • How does the method perform under severe group imbalance or inaccurate conditional density estimation?

  • Is flow matching really necessary here? What happens if the same regularizer is applied to a simpler deterministic prediction model with comparable capacity and training cost?

Reviewer_5cugOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The manuscript introduces CI-CFM, a conditional flow based approach to help enforce label-conditional fairness. A normalized density is also fitted on detached scores, and the reference is built by mixing the same components with the training set weights. The empirical pipeline alternates between fitting the density and updating the predictor. Experiments are performed on synthetic regression, Adult, MIMIC and CIFAR-10 and CelebA (but only under an independent-nuisance null control). The experiments on the vision datasets don't provide additional signal, the empirical results don't seem comprehensive and the empirical claims aren't well supported by the theory.

P.S. The link to the open-sourced code is broken, so the reproducibility is unverified.

Critical strengths

[S1]: Clear motivation and an interesting idea: The authors do a good job of devising a framework that overcomes the flaws of the two-head design, as discussed in App. A.4, and the general algorithmic design is cleverly designed to attack the problem the authors define. Resolving the mismatch between training-time stochastic regularization and inference-time predictions is well-motivated, and seems to work to reduce HSIC without sacrificing on the performance as seen in the outcomes in Tab. 1 and Tab. 2.

Critical weaknesses

[W1]: Backing for the experimental results is weak.: Theorem 3.1 is borrowed from Ahuja et al., and does not directly lend to any guarantees for the method proposed in any way whatsoever. The guarantees provided by Prop. A.3 are dependent on εcond\varepsilon_{cond}, which is not controlled or monitored by the alternating training procedure. Further, as the author admit in Line 916, the Prop. A.2 results don't hold in score tails, which is precisely where methods fail, and the log-ratios blow up. Thus, there is no backing for the convergence of the alternative training. More expansion on App. A.6 would have helped understand the practical outcomes better.

[W2]: Experimental results themselves seem incomplete.: In Tab. 10 highlights that Deterministic+CI has significantly lower dependence scores across all fairness metrics, but at the slight cost of the performance. The authors present results for only λ=0.2\lambda =0.2, and do not study the fairness-performance tradeoff in a Pareto curve by performing sweeps. It is unclear what the truly better approach is and severely undermines the proposed method. The results in Tab. 5 seem moot as the sensitive attribute is an independent Gaussian noise, and not an attribute like gender, hair color, age etc. Furthermore, as clarified in App. C.3 the regularization is applied every 8192 steps for CIFAR10 and every 1024 steps which is so infrequent that it is unclear what the effect of the regularization on fairness even is in high-dimensional data tasks. Also, in Tab.1 InfoNCE and CIRCE results are too identical, and very close to the baseline plain CFM backbone, which raises concerns about if the comparative methods were implemented properly, and if they were tuned at all.

Questions for authors

[Q1]: See [W1] for details. Can you tie the theory section to fit the implementation and empirical claims better? It would be helpful to see propositions or theorems which fit the regime the proposed approach will work under. Even if there is no guarantee for convergence, evidence of why it would be plausible would help strengthen the paper. What is the central theoretical claim, if any of the paper?

[Q2]: See [W2] for details. The empirical section needs significantly more comprehensiveness to convince me of the effectiveness of the proposed method. Pareto frontiers for Tab. 1, Tab. 5 with sweeps for parameters to ensure the baselines are performing the best they could and not a fixed λ=0.2\lambda=0.2 value (which is also not justified for all settings btw), would be more convincing to understand the tradeoff, and the robustness and effectiveness of the approach. Further details on what the vision based experiments imply would also help, as in the current form they don't say anything about the fairness-performance tradeoff for real-world sensitive attributes at all. Further, Line 1327 says that the random split for MIMIC is across ICU stays, so does this mean a patient could be in the train and test set both if they had multiple stays? Can you provide reassurance that results across all settings are indeed significant, and if so, what the takeaway is, in terms of detailed sweeps and trade-offs? What justifies using CI-CFM which is a lot more computationally expensive than any of these methods, or even the deterministic one when the results are not impressive or universal enough?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.