acceptodds
Under review as a conference paper at ICLR 2027

Domain Adapted Diffusion Models for Conditional Independence Testing

Abstract

Conditional independence (CI) is a fundamental concept in statistics and machine learning. Recent advances in conditional generative modeling provide flexible tools for generative-model-based CI tests, which rely on an estimated conditional distribution to generate randomized samples. However, errors in estimating this distribution accumulate in existing Type I error bounds, and consistency of the generative estimator alone does not guarantee asymptotic Type I error control. To address this limitation, we formulate conditional generative modeling as a domain adaptation problem and leverage auxiliary data from multiple source domains to improve estimation in the target CI testing domain. We propose Domain-Adapted Diffusion (DA-Diff), a multi-source domain adaptation framework for conditional diffusion models based on weighted empirical risk minimization over both target and source domains. We establish the convergence rate of DA-Diff and show how transferable source data can improve target-domain estimation through an increased effective sample size while controlling transfer bias. Building on DA-Diff, we further propose Domain-Adapted Conditional Independence Testing (DA-CIT) and show that its Type I error satisfies . Experiments demonstrate that DA-Diff improved conditional generation quality compared with transfer-learning diffusion baselines, while DA-CIT provides strong Type I error control and competitive power.

Primary area:
generative models
Keywords:
Generative models, Conditional independence test, Domain adaptation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_rn4BOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies conditional independence testing with limited target data and additional data from different source domains. It proposes DA-Diff, which trains a conditional diffusion model by weighting target and source data. The analysis separates estimation error from the bias caused by differences between domains. The learned model is then used in a conditional randomization test, called DA-CIT. Experiments cover synthetic data and the Sachs dataset.

Critical strengths
  • The paper addresses a useful problem. Extra data can improve conditional distribution estimation, but differences between domains can also make the resulting test less reliable.

  • The theory describes the trade-off between using more source data and introducing more bias. This is a useful way to understand multi-source diffusion models.

  • The experimental results are promising. DA-Diff improves over target-only training in Table 1. DA-CIT generally keeps the observed Type I error near the nominal level while maintaining competitive power. The paper also includes several baselines and sensitivity studies.

Critical weaknesses
  • It is unclear whether the theory covers the actual training procedure. Theorems 1–3 analyze an estimator with a given weight vector. In practice, the method repeatedly estimates source differences, selects weights using validation data, and updates the model. It also uses different weights at different diffusion times. Appendix K.8 analyzes the estimated discrepancy matrix, but assumes that the validation data are independent of the fitted models. In the actual procedure, validation data affect the weights, which then affect later models. I could not find a proof that handles this dependence. Showing that the discrepancy estimate is consistent does not, by itself, establish the stated guarantee for the full algorithm.

  • The conditions for asymptotic Type I error control are strong and differ from the experimental settings. Equation (32) requires source domains to become increasingly similar to the target as the sample size grows. It also requires the effective sample size to grow faster than the square of the target sample size. The main experiments instead use ten source domains, similar sample sizes across domains, and fixed shift strengths. Scaling this setup does not satisfy Equation (32). Some nonlinear transformations in Appendix D.2 also produce distributions outside Assumption 2. These are sufficient conditions, so failing to meet them does not mean the method fails. However, the current theory does not directly explain why the method works in the experiments. The paper should make this limitation clearer.

  • The testing experiments do not clearly show how much the proposed weighting helps. A useful comparison would use pooled or finetuned diffusion models inside the same CRT, with the same source data, target split, and test statistic. This would show whether the proposed weights improve testing beyond simpler ways of using extra data. The Cat-all experiments combine domains before testing, so they do not provide this controlled comparison. Also, each setting uses only 100 repetitions. At a true rejection rate of 0.05, the standard error is about 0.022. The results are encouraging, but they cannot precisely distinguish good calibration from a small increase in Type I error.

Questions for authors
  • Does the theory cover the full algorithm, including repeated use of validation data and weights that vary across diffusion time? If not, please state which version of the method is covered.

  • What guarantee remains when source–target differences stay fixed and source sample sizes grow at the same rate as the target sample size?

  • How does DA-CIT compare with the same target-domain CRT using pooled or finetuned diffusion models? Please also report uncertainty intervals for the estimated Type I error.

Reviewer_o6ZEOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Conditional independence (CI) is a fundamental concept in statistics and machine learning, and CI testing considers the hypotheses H0:X0⊥Y0∣Z0H_0: X_0 \perp Y_0 \mid Z_0 versus H1:X0⊥̸Y0∣Z0H_1: X_0 \not\perp Y_0 \mid Z_0. Generative-model-based CI tests, such as the Conditional Randomization Test (CRT), rely on an estimate P^0(⋅∣Z0)\hat P_0(\cdot \mid Z_0) of the conditional distribution P0(⋅∣Z0)P_0(\cdot \mid Z_0) to generate randomized samples of X0X_0. When target-domain data are limited, however, this estimated distribution may be inaccurate, which can lead to inflated Type I error. To address this, the authors propose Domain-Adapted Diffusion (DA-Diff) and Domain-Adapted Conditional Independence Testing (DA-CIT), which leverage KK external source datasets to improve estimation in the target domain. Domain discrepancies are measured in score-function space, and source-domain weights wkw_k are learned separately for different diffusion time bins, with the aim of increasing the effective sample size (NeffN_{\mathrm{eff}}) while controlling transfer bias. The paper provides theoretical results showing that DA-CIT controls Type I error asymptotically under certain conditions. Empirically, the method remains close to the nominal significance level α=0.05\alpha=0.05 for conditioning dimensions up to dz=100d_z=100, and achieves the highest F1 score among the compared methods on the multi-source Sachs dataset.

Critical strengths
  1. The paper explicitly connects the estimation error of the generative model to the validity of the CI test. As shown in Eq. 2, the conditional distribution estimation error is multiplied by n0n_0 in the Type I error bound, so the consistency of the generative model alone does not guarantee Type I error control. The authors show that the error of DA-Diff decomposes into two parts: an estimation term that decreases with the effective sample size NeffN_{\mathrm{eff}}, and a transfer-bias term governed by the source discrepancy matrix GG (Theorems 1--2). Under suitable conditions, they further prove that DA-CIT achieves asymptotic Type I error control (Theorem 3).

  2. The empirical calibration of the test is convincing. Figure 6 shows that the p-values of DA-CIT under the null hypothesis closely follow a uniform distribution, as expected of a valid test. DA-CIT is the only method that passes the KS test at every tested dzd_z, whereas every other method fails in two or more dimensions, and CDCIT fails at every dz≥20d_z \geq 20.

Critical weaknesses
  1. The authors acknowledge that the weights are chosen through a surrogate criterion (Appendix C) and that the time-binned weighting departs from the single-weight formulation of the theory (Appendix H). However, no guarantee is provided for the procedure actually used, in which the weights are data-dependent, updated during training, and obtained by solving a surrogate optimization problem whose hyperparameters are selected by a gradient-alignment criterion over finite candidate grids, rather than being the optimal weights w∗w^* in Eq. 13. This leaves a gap between the theoretical analysis and the method as implemented.

  2. Please move the Related Work section from Appendix A into the main text. The introduction reviews CI testing methods, but the comparison with transfer learning for diffusion models, domain adaptation, and weighted ERM is central to assessing the novelty of DA-Diff. Moving the Related Work section into the main text would make the novelty of DA-Diff easier to assess and substantially improve the paper’s presentation.

Questions for authors
  1. The source domains in the experiments of Section 4.2 have fixed distribution shifts relative to the target, while the guarantee requires the source-to-target discrepancy to decrease with n0n_0 at rate γ>1\gamma > 1. Since DA-CIT still remains well calibrated in these experiments (Figure 1), is this because the learned weights suppress the less transferable sources, or does it suggest that Type I error control holds under weaker conditions than those in Theorem 3?

  2. The theoretical analysis assumes a fixed weight vector, while the actual algorithm uses data-dependent weights that are updated during training and may differ across diffusion time bins. How does this difference affect Type I error control? For example, how does the Type I error change when a single fixed weight vector is used instead of time-binned, adaptively updated weights?

Reviewer_tyxJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies conditional distribution estimation and conditional independence testing across different domains. To leverage samples from different source domains while avoiding negative transfer caused by distribution shifts, the authors propose DA-Diff, which trains conditional diffusion models using weighted ERM. By analyzing the estimation error and transfer bias introduced by different source domains, the authors develop a corresponding source weighting mechanism. The authors theoretically establish connection between the proposed method and Type I error control in CRT, and demonstrate the experimental effectiveness of the method.

Critical strengths
  1. This paper investigates a meaningful problem: how to effectively leverage data from different source domains to improve conditional distribution estimation in the target domain while avoiding negative transfer.
  2. The authors provide a relatively systematic theoretical analysis. The overall theoretical framework is well developed and closely related to the problem studied in this paper.
  3. The paper is relatively complete. Based on the theoretical analysis, the authors develop a practical source weighting and training mechanism, and provide detailed descriptions of the experimental settings and implementation.
Critical weaknesses
  1. The independent contribution of the core methodological innovation needs further validation. The basic training framework of DA-Diff is based on the commonly used weighted ERM. From a methodological perspective, one of the main innovations of this paper is the weight selection mechanism based on conditional score discrepancy. Although the authors provide hyperparameter sensitivity analyses and other evaluations, the overall method also combines several other training strategies. The independent effectiveness of this core mechanism needs further validation. For example, the authors could compare the proposed weight selection method with simpler adaptive strategies while keeping other training strategies unchanged. In addition, as the true distributions are known in the synthetic experiments, weight selection based on the true score discrepancy could also serve as a reference to more directly validate the effectiveness of the proposed method.
  2. The improvement brought by the model itself needs to be better isolated. In Section 4.2, the authors mainly compare DA-CIT with other CIT methods. Given that finetune outperforms DA-Diff in some settings in table 1, I suggest comparing the conditional generative models obtained by DA-Diff and finetune under the same CRT framework, using metrics such as type I error. This would help evaluate the contribution of the proposed weighting method.

Minor 3. The type I error and power results in section 4.2 are mainly based on 100 repetitions. Could the authors consider adding confidence intervals or error bars to better show the stability of the results?

Questions for authors

My main concern is the first point in weaknesses. Since the main methodological innovation of this paper lies in the weight selection mechanism, I would like the authors to further explain how the current experiments validate its independent contribution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.