acceptodds
Under review as a conference paper at ICLR 2027

Doubly Robust Deep Kernel Tests for Structured Outcomes

Abstract

We propose a test of whether two interventions induce the same outcome distribution when outcomes are structured and high-dimensional, such as images, text and graphs. We represent the interventional distributions as mean embeddings on learned deep features, such that the treatment effect is a doubly robust representation of the difference in mean embeddings. Our test, DR-KTE-Deep, optimises the learned features so as to maximise an estimated power criterion on this treatment effect representation. To motivate our learning objective, we show that the power criterion characterises the asymptotic signal-to-noise ratio of the test under fixed alternatives, and we derive a uniform finite-sample bound for its regularised estimate. We show that, with the learned kernel fixed, a held-out test is asymptotically valid under the null given suitable nuisance-estimation rates. Empirically, DR-KTE-Deep achieves substantially higher power than fixed and bandwidth-optimised kernel baselines across image, text and graph outcomes, while maintaining well-calibrated Type-I error.

Primary area:
causal reasoning
Keywords:
kernel methods, hypothesis testing, causal inference, doubly robust estimation, deep kernels, kernel mean embeddings, distributional treatment effects, structured outcomes

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qhvhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether two interventions induce the same outcome distribution when outcomes are structured and high-dimensional, such as images, text, and graphs. The proposed method, DR-KTE-Deep, combines doubly robust estimation of interventional mean embeddings with deep kernel learning. The kernel is trained to maximize an estimated signal-to-noise criterion related to test power. A characteristic kernel component is retained so that the final kernel remains sensitive to general distributional differences.

The paper provides a doubly robust representation of the distributional treatment effect, asymptotic validity of a held-out test after kernel learning and nuisance estimation, and uniform finite-sample control of the kernel-learning criterion. Experiments on image, text, and graph outcomes generally show higher power than fixed Gaussian and bandwidth-optimized DR-KTE baselines while maintaining Type-I error near the nominal level.

Critical strengths
  1. The paper addresses an important problem. Many modern causal outcomes are images, text, or graphs, for which a scalar average treatment effect can miss important changes in the outcome distribution. The paper gives a general way to test changes in the full marginal interventional distribution without first reducing the outcome to a single scalar feature.

  2. The theoretical construction is careful. The paper separates kernel learning, validation, and final testing, which allows the final test to remain valid after data-dependent kernel selection. The characteristic component also preserves the omnibus interpretation of the test. The experiments cover several different types of structured outcomes and show large power improvements in several settings.

Critical weaknesses
  1. I am not yet convinced that the experiments establish the unique value of the proposed deep-kernel learning procedure relative to strong current alternatives. The main direct comparisons are DR-KTE with a fixed Gaussian kernel and DR-KTE-Opt with an optimized Gaussian bandwidth. These are natural predecessors, but they can be weak representations for images, text, and graphs. This is especially visible in the text ablation: once the text is represented with a strong fixed embedding, all three methods have almost identical power. The scalar experiments also contain settings where the optimized Gaussian kernel performs as well as or slightly better than the deep kernel. Therefore, part of the reported gain may come from replacing a weak fixed representation rather than from the proposed causal power-learning objective itself. The paper itself cites recent methods that learn causal outcome representations or informative witness locations, but does not compare against them empirically. These methods do not all test exactly the same omnibus null, so a direct comparison may require adaptation. Still, a stronger comparison or a clearer explanation of why these approaches are not comparable would make the contribution easier to assess. Durand et al. is a UAI 2025 paper on learning causal response representations, while related approaches from Wibisono and Wang and Zenati and Gretton appeared in 2026.

  2. There remains a gap between the theoretical robustness claims and the empirical validation. The structured-outcome experiments are synthetic or semi-synthetic, which is useful for measuring power but gives limited evidence about performance with realistic nuisance estimation, weak overlap, or complex observational confounding. The robustness experiment studies a misspecified outcome model while assuming the logging policy is known. It does not show the complementary case of a misspecified propensity model with a well-specified outcome model. This seems important for a method whose causal construction is based on double robustness.

The dimension-scaling experiment also shows that power eventually collapses and Type-I error rises above the nominal level for all methods as irrelevant outcome dimensions are added. I would like a clearer discussion of what this implies for genuinely high-dimensional applications.

Questions for authors
  1. How much of the empirical gain comes from the proposed power-learning objective rather than simply learning a better outcome representation? Could the authors compare DR-KTE against stronger fixed representations for each modality, such as pretrained image or text features? The text-embedding ablation is particularly important because the advantage of DR-KTE-Deep almost disappears there.

  2. Can the authors provide the complementary double-robustness experiment? Section D.5 studies outcome-model misspecification while using the true logging policy. What happens when the propensity model is misspecified but the conditional outcome model is well estimated? I would also be interested in the case where both nuisance models are imperfect.

  3. How realistic are the nuisance-rate assumptions for the estimators used in the experiments? The appendix states that the required learner rates are assumed rather than proved. Can the authors provide empirical evidence on calibration under weaker overlap or harder nuisance estimation, or explain when these conditions should be expected to hold in practice?

5 comments

We thank you for your thoughtful feedback. Below, we provide responses to your concerns and would be happy to offer further clarification if needed. Should you find our responses satisfactory, we would be grateful if you would consider increasing your score.

On representation learning

We provide additional experiments to assess the contribution of our power-learning criterion beyond fixed and unsupervised outcome representations.

Image outcomes

Appendix D.6 isolates the training objective: on OCT-Drusen, with the same encoder architecture, initial weights and data, DR-KTE-Deep achieves power 0.371 ±\pm 0.080, versus 0.057 ±\pm 0.004 (random features) and 0.052 ±\pm 0.004 (pixel autoencoder).

We also test stronger fixed representations. We embed each image using a frozen ResNet-18 pretrained on ImageNet, then apply DR-KTE, DR-KTE-Opt and DR-KTE-Deep to the same embeddings.

H1H_1 / H0H_0DR-KTEDR-KTE-OptDR-KTE-Deep
Breast-Tumor0.799 ±\pm 0.015 / 0.054 ±\pm 0.0030.799 ±\pm 0.015 / 0.054 ±\pm 0.0030.860 ±\pm 0.015 / 0.058 ±\pm 0.004
Chest-Cardiomegaly0.374 ±\pm 0.015 / 0.059 ±\pm 0.0030.375 ±\pm 0.015 / 0.059 ±\pm 0.0030.626 ±\pm 0.040 / 0.059 ±\pm 0.004
Chest-Effusion0.870 ±\pm 0.013 / 0.049 ±\pm 0.0030.870 ±\pm 0.013 / 0.048 ±\pm 0.0040.941 ±\pm 0.009 / 0.048 ±\pm 0.002
Chest-Mass0.807 ±\pm 0.017 / 0.059 ±\pm 0.0030.807 ±\pm 0.017 / 0.059 ±\pm 0.0030.901 ±\pm 0.014 / 0.059 ±\pm 0.004
Derma-Melanoma0.955 ±\pm 0.007 / 0.045 ±\pm 0.0040.955 ±\pm 0.007 / 0.044 ±\pm 0.0040.987 ±\pm 0.003 / 0.045 ±\pm 0.004
OCT-CNV0.980 ±\pm 0.004 / 0.047 ±\pm 0.0030.981 ±\pm 0.004 / 0.047 ±\pm 0.0030.997 ±\pm 0.001 / 0.050 ±\pm 0.004
OCT-DME0.612 ±\pm 0.023 / 0.047 ±\pm 0.0030.611 ±\pm 0.023 / 0.047 ±\pm 0.0030.802 ±\pm 0.025 / 0.050 ±\pm 0.004
OCT-Drusen0.444 ±\pm 0.019 / 0.051 ±\pm 0.0040.444 ±\pm 0.019 / 0.051 ±\pm 0.0040.724 ±\pm 0.030 / 0.052 ±\pm 0.004
Pneumonia0.895 ±\pm 0.010 / 0.053 ±\pm 0.0050.895 ±\pm 0.010 / 0.053 ±\pm 0.0050.938 ±\pm 0.009 / 0.051 ±\pm 0.004
Retina-Severe0.996 ±\pm 0.001 / 0.051 ±\pm 0.0040.996 ±\pm 0.001 / 0.050 ±\pm 0.0040.998 ±\pm 0.001 / 0.056 ±\pm 0.004

Compared with Table 1 of the submission, pretrained ResNet features improve the estimated power of both Gaussian-kernel baselines over raw pixels in all ten scenarios. Yet on the same pretrained representation, DR-KTE-Deep has higher estimated power in every scenario, with similar Type-I error. Thus, its gains are not solely due to replacing raw pixels with informative pretrained features.

Text outcomes

In Appendix D.7, pretrained sentence embeddings already make the original sentiment and clinical-outcome shifts easy to detect, leaving little room for kernel learning. We therefore introduce Clinical Notes (Style), where treatment shifts writing toward telegraphic or longer, flowing prose rather than ordinary prose, while prompting the generator to preserve clinical content. This is a less directly encoded difference for semantic embeddings.

H1H_1 / H0H_0DR-KTEDR-KTE-OptDR-KTE-Deep
Clinical Notes (Style)0.426 ±\pm 0.021 / 0.046 ±\pm 0.0040.425 ±\pm 0.021 / 0.046 ±\pm 0.0040.622 ±\pm 0.046 / 0.051 ±\pm 0.004

On this style-shift scenario, the fixed-kernel tests have substantially lower estimated power than in the original. This supports the value of learning the kernel when the treatment-related difference is less well represented by the pretrained embedding. Moreover, DR-KTE-Deep trained directly on raw tokens achieves power comparable to DR-KTE on pretrained embeddings in Oncology Notes (0.809 ±\pm 0.023 versus 0.828 ±\pm 0.035) and Clinical Notes (0.939 ±\pm 0.013 versus 0.894 ±\pm 0.027).

Scalar outcomes

On scalar outcomes, which are not the focus of the method, DR-KTE-Opt uses the same power-learning criterion as DR-KTE-Deep but optimises only the Gaussian bandwidth. Its advantage in Scenario III is about one standard error (0.653 ±\pm 0.027 against 0.614 ±\pm 0.028).

Additional baselines

We add DR-ME (Zenati and Gretton, 2026) and adapt DEA (Durand et al., 2025) and MCF (Wibisono and Wang, 2026). Neither provides a drop-in test of our marginal interventional null: DEA learns outcome projections using a conditional-dependence regression criterion, which can miss mean-preserving shifts, whereas MCF learns a scalar causal-contrast feature without our variance normalisation. To compare them under the same null and testing protocol, we evaluate their learned features in our doubly robust held-out cross-U test, yielding DR-DEA and DR-MCF. These are testing adaptations, not the original procedures.

H1H_1 / H0H_0DR-KTEDR-MEDR-MCFDR-DEADR-KTE-Deep
OCT-Drusen0.051 ±\pm 0.003 / 0.058 ±\pm 0.0040.044 ±\pm 0.003 / 0.045 ±\pm 0.0040.318 ±\pm 0.049 / 0.053 ±\pm 0.0030.053 ±\pm 0.010 / 0.059 ±\pm 0.0080.371 ±\pm 0.080 / 0.058 ±\pm 0.005
Derma-Melanoma0.171 ±\pm 0.009 / 0.051 ±\pm 0.0020.400 ±\pm 0.029 / 0.045 ±\pm 0.0040.795 ±\pm 0.041 / 0.049 ±\pm 0.0040.048 ±\pm 0.004 / 0.053 ±\pm 0.0040.898 ±\pm 0.040 / 0.053 ±\pm 0.004
Chest-Cardiomegaly0.156 ±\pm 0.007 / 0.050 ±\pm 0.0030.178 ±\pm 0.009 / 0.046 ±\pm 0.0020.368 ±\pm 0.048 / 0.048 ±\pm 0.0040.043 ±\pm 0.004 / 0.044 ±\pm 0.0050.928 ±\pm 0.046 / 0.051 ±\pm 0.004

DR-KTE-Deep has higher estimated power than DR-MCF on all three scenarios. DR-DEA remains near nominal rejection under the alternatives, consistent with the limited regression signal in our mean-preserving image constructions. Our criterion instead targets the studentised distributional discrepancy (Proposition 4.1).

Robustness to misspecification

We test complementary nuisance misspecification at N=1500N=1500 (500 learning, 500 validation and 500 test observations). The misspecified propensity omits nonlinear treatment-assignment features; the misspecified CME uses a linear input kernel instead of features capturing the nonlinear outcome mechanism:

H1H_1 / H0H_0Both well-specifiedMisspecified μ^\hat\muMisspecified π^\hat\piBoth misspecified
Causal-Blob0.267 ±\pm 0.033 / 0.058 ±\pm 0.0110.211 ±\pm 0.030 / 0.055 ±\pm 0.0090.599 ±\pm 0.053 / 0.292 ±\pm 0.0590.935 ±\pm 0.029 / 0.735 ±\pm 0.054
Scenario II0.860 ±\pm 0.032 / 0.046 ±\pm 0.0040.510 ±\pm 0.043 / 0.045 ±\pm 0.0060.980 ±\pm 0.008 / 0.155 ±\pm 0.0390.995 ±\pm 0.003 / 0.865 ±\pm 0.033
Scenario III0.268 ±\pm 0.024 / 0.046 ±\pm 0.0040.192 ±\pm 0.015 / 0.045 ±\pm 0.0060.702 ±\pm 0.023 / 0.155 ±\pm 0.0390.833 ±\pm 0.022 / 0.865 ±\pm 0.033
Scenario IV0.893 ±\pm 0.017 / 0.046 ±\pm 0.0040.782 ±\pm 0.025 / 0.045 ±\pm 0.0060.994 ±\pm 0.002 / 0.155 ±\pm 0.0390.993 ±\pm 0.002 / 0.865 ±\pm 0.033
Fake-MNIST0.330 ±\pm 0.044 / 0.048 ±\pm 0.0050.309 ±\pm 0.043 / 0.055 ±\pm 0.0040.604 ±\pm 0.054 / 0.146 ±\pm 0.0190.812 ±\pm 0.029 / 0.454 ±\pm 0.016

With a richer propensity model, misspecifying the CME leaves Type-I error near nominal. A richer CME substantially reduces the inflation from propensity misspecification, but does not fully restore calibration at this sample size; misspecifying both nuisances causes much greater inflation.

Proposition 3.3 gives double robustness of the population contrast, not automatically of test calibration. Theorem 4.3 requires sufficiently small null bias, ensured by a nuisance-error product oP(m−1/2)o_P(m^{-1/2}); this need not hold when propensity misspecification persists and the CME is estimated nonparametrically (Benkeser et al., 2017; Dukes et al., 2024).

To test whether a better outcome model reduces this inflation, we keep the misspecified propensity, learned kernel and test set fixed, and refit the CME on 4n4n and 16n16n observations (dividing its ridge penalty by 2 and 4):

H1H_1 / H0H_0nn4n4n16n16n
Causal-Blob0.599 ±\pm 0.053 / 0.292 ±\pm 0.0590.602 ±\pm 0.027 / 0.094 ±\pm 0.0100.575 ±\pm 0.038 / 0.047 ±\pm 0.004
Scenario III0.702 ±\pm 0.023 / 0.155 ±\pm 0.0390.744 ±\pm 0.030 / 0.071 ±\pm 0.0050.752 ±\pm 0.026 / 0.066 ±\pm 0.005

Improving the CME markedly reduces Type-I inflation: calibration becomes close to nominal on Causal-Blob, though some inflation remains in Scenario III, with broadly similar estimated power.

With an oracle propensity, fitting the CME also improves Scenario III power (0.616 versus 0.349; Appendix D.5), with similar Type-I error.

Nuisance-rate assumptions

For n≍mn\asymp m, a correctly specified root-nn propensity estimator and any consistent CME suffice for the product-rate condition of Theorem 4.3 (Chernozhukov et al., 2018). Our experiments fall within this favorable regime: treatment assignment follows the logistic family used for propensity estimation, which admits a parametric rate under standard regularity conditions. The remaining requirement is CME consistency, for which KRR has established sufficient conditions. More generally, two nonparametric estimators with root-mean-square rates n−s/(2s+dX)n^{-s/(2s+d_X)} suffice when s>dX/2s>d_X/2. Hilbert-valued KRR can attain such rates under suitable smoothness and regularisation conditions, without an exponent depending directly on outcome dimension (Li et al., 2024).

We stress-test overlap by scaling the treatment-assignment coefficients and widening the true propensity clipping bounds (N=1500N=1500, with 500 learning, 500 validation and 500 test observations):

Propensity boundsPropensities at the boundsPropensity errorEffective sample sizeH1H_1 / H0H_0
Causal-Blob, α\alpha[0.07,0.93][0.07, 0.93]4%0.02459%0.382 ±\pm 0.024 / 0.050 ±\pm 0.004
Causal-Blob, 1.5α1.5\alpha[0.07,0.93][0.07, 0.93]21%0.03030%0.130 ±\pm 0.012 / 0.044 ±\pm 0.004
Causal-Blob, 2α2\alpha[0.07,0.93][0.07, 0.93]36%0.04416%0.072 ±\pm 0.007 / 0.048 ±\pm 0.005
Causal-Blob, 2α2\alpha[0.01,0.99][0.01, 0.99]8%0.02225%0.259 ±\pm 0.022 / 0.122 ±\pm 0.014
Causal-Blob, 3α3\alpha[0.001,0.999][0.001, 0.999]8%0.02223%0.467 ±\pm 0.018 / 0.238 ±\pm 0.033
OCT-Drusen, α\alpha[0.07,0.93][0.07, 0.93]7%0.03450%0.371 ±\pm 0.080 / 0.058 ±\pm 0.005
OCT-Drusen, 1.5α1.5\alpha[0.07,0.93][0.07, 0.93]22%0.04020%0.198 ±\pm 0.048 / 0.042 ±\pm 0.003
OCT-Drusen, 2α2\alpha[0.07,0.93][0.07, 0.93]36%0.05210%0.045 ±\pm 0.005 / 0.034 ±\pm 0.003

With bounds 0.07,0.930.07,0.93, weaker overlap reduces effective sample size and power but leaves null rejection near nominal. Widening the bounds produces finite-sample Type-I inflation, which Theorem 4.3 does not rule out.

Dimension scaling experiment

Appendix D.11 adds independent noise coordinates while keeping the treatment-related difference in the first two dimensions. All three kernel tests lose power, but DR-KTE-Deep retains an advantage through d=16d=16.

We traced the previously reported Type-I inflation to an implementation error in this experiment. After correction, all three displayed kernel tests have null rejection rates near 5% across dimensions.

H1H_1 / H0H_0DR-KTEDR-KTE-OptDR-KTE-Deep
d=2d=20.849 ±\pm 0.013 / 0.046 ±\pm 0.0030.910 ±\pm 0.011 / 0.048 ±\pm 0.0030.973 ±\pm 0.009 / 0.043 ±\pm 0.004
d=3d=30.528 ±\pm 0.019 / 0.048 ±\pm 0.0040.731 ±\pm 0.019 / 0.052 ±\pm 0.0040.935 ±\pm 0.025 / 0.050 ±\pm 0.003
d=4d=40.292 ±\pm 0.012 / 0.053 ±\pm 0.0040.536 ±\pm 0.022 / 0.048 ±\pm 0.0040.920 ±\pm 0.028 / 0.042 ±\pm 0.003
d=8d=80.091 ±\pm 0.005 / 0.051 ±\pm 0.0040.155 ±\pm 0.016 / 0.048 ±\pm 0.0030.507 ±\pm 0.073 / 0.050 ±\pm 0.004
d=16d=160.058 ±\pm 0.004 / 0.049 ±\pm 0.0040.059 ±\pm 0.004 / 0.054 ±\pm 0.0040.119 ±\pm 0.021 / 0.052 ±\pm 0.004
d=32d=320.050 ±\pm 0.004 / 0.047 ±\pm 0.0030.054 ±\pm 0.004 / 0.050 ±\pm 0.0040.050 ±\pm 0.003 / 0.051 ±\pm 0.004

We will update Appendix D.11 accordingly.

Reviewer_Aiu6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a statistical test to check whether a treatment changes structured, high-dimensional outcomes. They approach this problem using observational datasets where each observation is a 3-tuple (X, A, Y). X affects both A and Y, and acts as a sufficient adjustment set. The authors correct for selection bias using a doubly robust estimator. It combines two helper models: one predicts how likely each patient was to be treated, and the other predicts what each patient's outcome would look like under each treatment. Tey combine these two models using a doubly robust estmator that works for ATE. They then learn the kernel features using a neural network. The network is trained in an end-to-end manner with an objective that aims to maximise the statistical test's signal-to-noise ratio.

Critical strengths
  1. The paper appears to be well written, although I could not follow the dense theoretical contributions largely. But for a person not from this field, I could get the overall idea.
  2. I appreciate the fact that the authors attempted the probem in high dimensional outcome spaces which is very challenging.
  3. The experimental setup and other details are discussed to adequate level of details.
Critical weaknesses

The problem formulation seems fine to me. I am not from kernels background so I leave it to the wisdom of other reviewers to assess the novelty of the proposed kernel statistical test.

From a causal inference perspective, particularly when I view the paper from treatment effect estimation perspective, I have several concerns in the experiments.

Questions for authors
  1. In real life X should affect Y. But in the synthetic experimets, X seems to have no causal effect on Y. It seems to affect only through the \delta. How do we connect thin in practice?

  2. Unconfoundedness given X seems to be a very hard assumption. In the X-ray case only, can we justify why this would hold in real life? Doctors typically decide treatment using symptoms, past history, and what they see in the scan itself, and much of this is not in X.

  3. In this experiment, X is just random numbers and is not taken from the image. In real life, the things that decide treatment are often visible in the image itself. How would the method work in that case?

  4. Under the null, the image does not depend on X, so even a test that ignores confounding would just have about 5% false positives. How do the reported Type-I errors show that the confounding correction works?

  5. In every experiment, the propensity model has the same form as the true treatment rule. Can the authors show results when the propensity model is wrong? In real life, the DR property is not that useful because it is very hard to obtaion calibrated probability measures out. Infact, this is the reason why in treatment effect methods, oftentime, just potential outcome based methods outperform methods that involve propensity estimates.

  6. For images, the baseline uses a Gaussian kernel on raw pixels, and for text it uses raw token ID numbers. Why not compare against features from a pretrained model, such as an ImageNet or medical image model, or a sentence embedding for text?

  7. Please include experiments using simple methods such as aggregated kernel tests (MMDAgg, MMD-FUSE) and also include the learned witness method of Zenati and Gretton (2026)?

  8. How does a simple test that ignores confounding, comparing treated and untreated images directly, perform in these experiments?

  9. Can the authors compare the deep kernel with propensity weighting only, and with the outcome model only, on images?

  10. The lesion is either present or absent, which a CNN can detect easily. How does the method perform when the effect is spread over the whole image, such as a change in texture or brightness?

  11. The safeguard that keeps the kernel able to detect any difference uses a very small mixing weight of 1e-9. Is this even consequential in practice? What happens when the safeguard is dropped altogether?

5 comments

We thank you for your attentive review. We have carefully considered your points and provide answers below. We hope our answers will support a more favourable evaluation.

Adjustment for covariates

In the main medical-image experiments indeed, YY depends on XX only under H1H_1. We will fix this. Note however that Appendix D.1 includes image experiments where XX affects YY under both hypotheses. In Fake-MNIST, for example, the digit XX determines the outcome image and influences treatment assignment. Moreover, in the text experiment, XX shifts the tone of the text under both treatments. In the graph experiments, XX is computed from the pre-treatment graph, used as the outcome under H0H_0. All experiments demonstrate a well calibrated Type-I error under H0H_0.

We include an additional experiment where we ignore confounding on Fake-MNIST with a naive MMD test. We also include three medical image experiments modified such that the baseline image BiB_i scales with XiX_i, so the image is dependent on XX even under H0H_0.

H1H_1 / H0H_0DR-KTEDR-KTE-OptDR-KTE-DeepNaive MMD
Fake-MNIST0.225 ±\pm 0.008 / 0.051 ±\pm 0.0030.229 ±\pm 0.008 / 0.052 ±\pm 0.0030.380 ±\pm 0.050 / 0.050 ±\pm 0.0040.671 ±\pm 0.026 / 0.600 ±\pm 0.016
OCT-Drusen (modified)0.056 ±\pm 0.003 / 0.050 ±\pm 0.0030.055 ±\pm 0.003 / 0.049 ±\pm 0.0030.528 ±\pm 0.088 / 0.051 ±\pm 0.0030.782 ±\pm 0.041 / 0.535 ±\pm 0.017
Derma-Melanoma (modified)0.148 ±\pm 0.007 / 0.051 ±\pm 0.0030.148 ±\pm 0.007 / 0.051 ±\pm 0.0030.749 ±\pm 0.073 / 0.051 ±\pm 0.0020.993 ±\pm 0.002 / 0.980 ±\pm 0.002
Pneumonia (modified)0.817 ±\pm 0.018 / 0.056 ±\pm 0.0060.834 ±\pm 0.018 / 0.055 ±\pm 0.0070.998 ±\pm 0.001 / 0.056 ±\pm 0.0061.000 ±\pm 0.000 / 0.977 ±\pm 0.004

The naive MMD test uses the same learned kernel as DR-KTE-Deep. Ignoring confounding rejects between 54% and 98% of null datasets, while all doubly robust tests remain calibrated and DR-KTE-Deep has the highest power among them.

Our framework allows XX to include pretreatment images or features extracted from them, alongside other clinical covariates. Only information from the post-treatment outcome YY cannot be used for pretreatment adjustment.

Unconfoundedness

This is a substantive identification assumption. Our method addresses complex, structured outcomes. Unobserved confounding is a separate challenge and we do not conflate the two problems.

In the X-ray setting, XX would need to capture the relevant pretreatment common causes of treatment and outcome, such as symptoms, clinical history and disease severity, potentially including pretreatment imaging. Our Oncology Notes experiment illustrates such clinical covariates. We do not claim unconfoundedness holds in real clinical data; it holds by construction in our semi-synthetic experiments and by design in randomized trials.

Misspecified propensity

We will include this experiment in the updated manuscript. The true propensity is chosen to be nonlinear in xx, while the misspecified propensity is a logistic regression linear in xx. The misspecified outcome model is a CME with a linear kernel on xx, while the well-specified models include the nonlinear features.

H1H_1 / H0H_0Both well-specifiedMisspecified μ^\hat\muMisspecified π^\hat\piBoth misspecified
Causal-Blob0.267 ±\pm 0.033 / 0.058 ±\pm 0.0110.211 ±\pm 0.030 / 0.055 ±\pm 0.0090.599 ±\pm 0.053 / 0.292 ±\pm 0.0590.935 ±\pm 0.029 / 0.735 ±\pm 0.054
Scenario II0.860 ±\pm 0.032 / 0.046 ±\pm 0.0040.510 ±\pm 0.043 / 0.045 ±\pm 0.0060.980 ±\pm 0.008 / 0.155 ±\pm 0.0390.995 ±\pm 0.003 / 0.865 ±\pm 0.033
Scenario III0.268 ±\pm 0.024 / 0.046 ±\pm 0.0040.192 ±\pm 0.015 / 0.045 ±\pm 0.0060.702 ±\pm 0.023 / 0.155 ±\pm 0.0390.833 ±\pm 0.022 / 0.865 ±\pm 0.033
Scenario IV0.893 ±\pm 0.017 / 0.046 ±\pm 0.0040.782 ±\pm 0.025 / 0.045 ±\pm 0.0060.994 ±\pm 0.002 / 0.155 ±\pm 0.0390.993 ±\pm 0.002 / 0.865 ±\pm 0.033
Fake-MNIST0.330 ±\pm 0.044 / 0.048 ±\pm 0.0050.309 ±\pm 0.043 / 0.055 ±\pm 0.0040.604 ±\pm 0.054 / 0.146 ±\pm 0.0190.812 ±\pm 0.029 / 0.454 ±\pm 0.016

With a well-specified propensity, misspecifying the CME leaves Type-I error close to nominal. With a misspecified propensity, a richer CME substantially reduces Type-I inflation, although calibration is not fully restored at this sample size. Misspecifying both nuisances leads to much larger inflation. This distinction between double robustness of the contrast and valid inference is further discussed in our response to reviewer qhvh.

Pretrained embeddings

We ran all image experiments on frozen features from an ImageNet-pretrained ResNet-18, under the same testing protocol. Pretrained features improve the power of the fixed-kernel baselines over raw pixels, but DR-KTE-Deep outperforms them on all ten datasets with similar Type-I error, for example 0.724 ±\pm 0.030 against 0.444 ±\pm 0.019 on OCT-Drusen and 0.626 ±\pm 0.040 against 0.374 ±\pm 0.015 on Chest-Cardiomegaly. More details are given in our answer to reviewer qhvh.

For text, Appendix D.7 shows comparable power across methods using Qwen3-Embedding-0.6B features, as the original sentiment and clinical-content differences are well captured by pretrained embeddings. We add a more challenging style-shift scenario in the answer to reviewer qhvh, where DR-KTE-Deep achieves higher estimated power (0.622 versus 0.426 for DR-KTE), with Type-I error close to nominal.

Appendix D.6 isolates the training objective by comparing the same neural architecture to a pixel autoencoder objective and random weights. Our learning objective outperforms both baselines.

MMDAgg, MMD-FUSE and DR-ME baselines

MMDAgg (Schrab et al., 2023) and MMD-FUSE (Biggs et al., 2023) are designed for two-sample testing, and their original permutation calibration does not directly apply to observational causal testing. We therefore construct adaptations using our doubly robust statistics across Gaussian bandwidths, combined through a maximum or soft maximum, with Gaussian calibration accounting for their dependence. These adaptations do not inherit the original theoretical guarantees, but exhibit Type-I error close to nominal in our experiments. We also include DR-ME (Zenati and Gretton, 2026).

H1H_1 / H0H_0DR-KTEDR-MMDAggDR-MMD-FUSEDR-MEDR-KTE-Deep
OCT-Drusen0.051 ±\pm 0.003 / 0.058 ±\pm 0.0040.077 ±\pm 0.007 / 0.052 ±\pm 0.0040.067 ±\pm 0.006 / 0.052 ±\pm 0.0060.044 ±\pm 0.003 / 0.045 ±\pm 0.0040.371 ±\pm 0.080 / 0.058 ±\pm 0.005
Derma-Melanoma0.171 ±\pm 0.009 / 0.051 ±\pm 0.0020.217 ±\pm 0.015 / 0.057 ±\pm 0.0060.189 ±\pm 0.013 / 0.056 ±\pm 0.0040.400 ±\pm 0.029 / 0.045 ±\pm 0.0040.898 ±\pm 0.040 / 0.053 ±\pm 0.004
Chest-Cardiomegaly0.156 ±\pm 0.007 / 0.050 ±\pm 0.0030.179 ±\pm 0.015 / 0.049 ±\pm 0.0060.145 ±\pm 0.012 / 0.049 ±\pm 0.0040.178 ±\pm 0.009 / 0.046 ±\pm 0.0020.928 ±\pm 0.046 / 0.051 ±\pm 0.004

DR-KTE-Deep is the most powerful test on all tasks with well calibrated Type-I error. More details on the baseline adaptations will be added to the manuscript.

IPW-Cross and Plug-in estimator comparison

Appendix D.4 compares the doubly robust estimator with propensity weighting on scalar outcomes. We extend this comparison to image outcomes and include an outcome-only (g-computation) plug-in estimator:

H1H_1 / H0H_0DR-KTE-DeepIPW-CrossPlug-in
OCT-Drusen0.371 ±\pm 0.080 / 0.058 ±\pm 0.0050.282 ±\pm 0.060 / 0.081 ±\pm 0.0081.000 ±\pm 0.000 / 1.000 ±\pm 0.000
Derma-Melanoma0.898 ±\pm 0.040 / 0.053 ±\pm 0.0040.740 ±\pm 0.053 / 0.063 ±\pm 0.0051.000 ±\pm 0.000 / 1.000 ±\pm 0.000
Chest-Cardiomegaly0.928 ±\pm 0.046 / 0.051 ±\pm 0.0040.890 ±\pm 0.053 / 0.075 ±\pm 0.0071.000 ±\pm 0.000 / 1.000 ±\pm 0.000

Only DR-KTE-Deep maintains Type-I error close to nominal. The key distinction is nuisance sensitivity and is very important: the doubly robust score has a product-rate centring error, whereas IPW and g-computation retain first-order propensity and CME errors, respectively. These errors need not be oP(m−1/2)o_P(m^{-1/2}), as required for our Gaussian calibration, even with root-nn propensity estimation when n≍mn\asymp m (Martinez-Taboada et al., 2023). Moreover, under our image nulls, the oracle g-computation score vanishes, violating the required nondegeneracy condition (same issue as MMD degeneracy under the null). Thus, the same studentisation does not automatically provide valid inference for these plug-in variants. This explains the importance of the doubly robust construction for our precise test, without implying that IPW or outcome-only inference is generally impossible. Zenati and Gretton (2026) report the same over-rejection for their plug-in variant.

Diffuse treatments effects

We include below results on a mean-matched perturbation that affects image intensity and contrast across the whole image:

H1H_1 / H0H_0DR-KTEDR-KTE-OptDR-KTE-Deep
OCT-Drusen, intensity0.183 ±\pm 0.011 / 0.049 ±\pm 0.0040.183 ±\pm 0.011 / 0.049 ±\pm 0.0040.792 ±\pm 0.066 / 0.050 ±\pm 0.004
OCT-Drusen, contrast0.051 ±\pm 0.004 / 0.049 ±\pm 0.0040.050 ±\pm 0.004 / 0.050 ±\pm 0.0040.200 ±\pm 0.058 / 0.053 ±\pm 0.004
Derma-Melanoma, intensity0.237 ±\pm 0.012 / 0.055 ±\pm 0.0040.244 ±\pm 0.011 / 0.054 ±\pm 0.0040.673 ±\pm 0.071 / 0.050 ±\pm 0.004
Derma-Melanoma, contrast0.288 ±\pm 0.015 / 0.054 ±\pm 0.0040.288 ±\pm 0.016 / 0.053 ±\pm 0.0040.770 ±\pm 0.041 / 0.050 ±\pm 0.004
Pneumonia, intensity0.227 ±\pm 0.013 / 0.055 ±\pm 0.0050.232 ±\pm 0.013 / 0.054 ±\pm 0.0040.417 ±\pm 0.043 / 0.056 ±\pm 0.005
Pneumonia, contrast0.082 ±\pm 0.006 / 0.055 ±\pm 0.0050.081 ±\pm 0.006 / 0.055 ±\pm 0.0050.187 ±\pm 0.026 / 0.052 ±\pm 0.005

DR-KTE-Deep has higher power than the baselines. Type-I error is between 0.049 and 0.056 for all methods.

Characteristic safeguard

The characteristic component provides a theoretical guarantee that the learned kernel remains capable of detecting general distributional differences. Without this component, a non-injective neural feature map may discard directions along which the interventional distributions differ. The minimum weight 10−910^{-9} is primarily a theoretical safeguard, not a claim of substantial finite-sample power contributed by that component.

With wmin⁡=10−9w_{\min} = 10^{-9}, the learned weight of the characteristic component ends between 0.45 and 0.50, from an initial value of 0.5, so the minimum is never reached. The value 10−910^{-9} is therefore not consequential for power.

We include an experiment removing the characteristic safeguard:

H1H_1 / H0H_0wmin⁡=10−9w_{\min} = 10^{-9}No characteristic component
OCT-Drusen0.371 ±\pm 0.080 / 0.058 ±\pm 0.0050.420 ±\pm 0.085 / 0.056 ±\pm 0.003
Derma-Melanoma0.898 ±\pm 0.040 / 0.053 ±\pm 0.0040.888 ±\pm 0.049 / 0.055 ±\pm 0.004

Removing the characteristic component yields similar estimated power and Type-I error on both tasks. Thus, the safeguard preserves the omnibus guarantee without materially affecting empirical performance in these experiments.

Reviewer_pRmfOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes two concepts: a doubly robust kernel test that captures the interventional distribution in RKHS and a deep kernel approach to facilitate this test for high-dimensional and structured data. The joint objective of these contributions is to estimate treatment effects in high-dimensional and structured data where the commonly used characteristic kernels are insufficient.

Critical strengths
  1. The ideas developed in the paper are theoretically grounded and may be useful for future works along this line.
  2. The overall problem statement has useful applications.
Critical weaknesses
  1. The writing needs to be more accessible. In its current form, the paper jumps into the details without explaining the practical/intuitive implications of the assumptions and theorems. For example, Assumption 3.1 lists three conditions. But what do they mean in practice? How can a practitioner adjust their data collection procedure to satisfy these requirements? Please provide intuitive examples of how these assumptions work in practice.

  2. My major concern is with the overall novelty of this work. While the high-level problem statement is useful, the solution seems to be composed from existing papers and preprints. For example, the test statistic is strongly related to the one in (Zenati et al., 2025), the deep kernels idea seems to have taken quite a bit from prior works (Gretton et al., 2012b; Liu et al., 2020), and the uniform feasible learning criterion from (Zenati and Gretton, 2026).

Questions for authors

Can the authors clearly differentiate the submission from the works mentioned in "Weaknesses"?

1 comment

Thank you for your positive and detailed feedback. We answer your two concerns below.

  1. The writing needs to be more accessible. [...] For example, Assumption 3.1 lists three conditions. But what do they mean in practice? [...]

We will add clarifications and more intuitive explanations to the manuscript. The three conditions of Assumption 3.1 are the standard identification assumptions of the causal inference literature. None of them are introduced by deep kernel learning or any specificity of our method.

The first condition, consistency, assumes that the observed outcome corresponds to the potential outcome under the received treatment. For example, the recorded treatment is the one the patient actually received. The second condition, conditional exogeneity, assumes that the observed covariates XX, measured pre-treatment, suffice to adjust for treatment selection. This is the assumption behind backdoor adjustment, a widely used identification strategy for observational data. It is strong and the most often violated of the three. Instrumental and proxy variables can be used when confounders are unobserved, which we leave to future work. The last condition, positivity, assumes that every treatment compared has a non-zero probability of being given to every type of patient.

These three conditions hold by construction in our semi-synthetic experiments. In observational data, overlap problems can be detected from the estimated propensity scores and overlap diagnostics. Consistency depends on how the treatment is defined and recorded. Conditional exogeneity cannot be verified from observational data alone.

  1. My major concern is with the overall novelty of this work. [...]

Can the authors clearly differentiate the submission from the works mentioned in "Weaknesses"?

Our work builds on the doubly robust representation of Zenati et al. (2025) and on choosing a kernel to maximise test power (Gretton et al., 2012b; Liu et al., 2020).

The main methodological novelty lies in learning a powerful kernel to compare interventional distributions when no samples from these distributions are available. Existing deep kernel two-sample tests optimise a discrepancy between two observed distributions. In our setting, observed treatment groups do not follow the interventional distributions (see our answer to reviewer Aiu6 on adjustment for covariates). Our method therefore optimises a power criterion built from doubly robust interventional mean embeddings, a challenge that existing two-sample tests do not face. The resulting algorithm combines nuisance estimation, causal power optimisation, kernel selection and a held-out test with a valid threshold. It is not a neural network substituted for a Gaussian kernel.

Our main theoretical contribution is Theorem 4.4, which controls the feasible criterion uniformly over the kernel class, accounting for both nuisance errors and sampling complexity. Its sampling term is O(log⁡m/m)O(\sqrt{\log m / m}) for a regulariser λ≥λ0>0\lambda \ge \lambda_0 > 0, against the O(m−1/3)O(m^{-1/3}) stated in Theorem 1 of Gretton et al. (2012b). Its discrepancy numerator retains the doubly robust product remainder ϵπϵμ\epsilon_\pi \epsilon_\mu, while estimating the variance introduces additional terms ϵπ\epsilon_\pi and ϵμ2\epsilon_\mu^2. Theorem 4.3 establishes that the test remains asymptotically valid after data-dependent kernel selection.

Finally, our method focuses on structured outcomes, with experiments on images, text and graphs, while retaining a characteristic component so that the test remains omnibus.

Zenati and Gretton (2026) learn a finite set of interpretable witness evaluations with a fixed kernel. While their test controls Type-I error under the global null, it is not guaranteed to detect every distributional alternative. In contrast, our learned kernel remains characteristic, preserving sensitivity to all differences between interventional distributions. This distinction matters in applications such as medical imaging or genomics, where the goal is to detect any change in the outcome distribution, without restricting attention to a finite set of witness locations.

Our work makes a methodological contribution to principled representation learning for causal testing, supported by statistical guarantees, which we believe aligns well with ICLR. Its novelty lies in the complete learning and testing framework, rather than in any individual component considered in isolation.

Comments on the paper

1 comment

Dear reviewers,

Thank you for your careful reviews. We address your comments in our individual responses, and summarise below the updates to the manuscript, marked in blue.

  • Learning representations on our objective improves on pretrained representations (Appendix D.7, D.8). On frozen ResNet-18 image embeddings and on text embeddings where treatment affects style rather than content, DR-KTE-Deep has higher power than the Gaussian baselines on the same representation, with well-calibrated Type-I error. The gain therefore comes from learning the representation for power.
  • Additional Baselines (Appendix D.16). DR-KTE-Deep has the highest power against DR-ME and adaptations of MMDAgg, MMD-FUSE, MCF and DEA.
  • Writing-Style and Diffuse Treatment Effects (Table 2, Appendix D.1). DR-KTE-Deep has the highest power on treatment effects affecting the writing style of a text outcome, no matter the representation, and on treatment effects affecting the whole image outcome rather than a localised section.
  • Estimator Comparison (Appendix D.4). Against IPW, outcome-only plug-in and kernel policy tests, DR-KTE-Deep is the only estimator with Type-I error close to the nominal level in all scenarios.
  • Effect of the Characteristic Component (Appendix D.17). We compare DR-KTE-Deep with and without the characteristic component of the kernel, and find that the characteristic safeguard has no meaningful empirical impact on power, and thus remains a theoretical guarantee.
  • Robustness to Nuisance Misspecification (Appendix D.5). With a misspecified propensity model, a well-specified conditional outcome embedding reduces Type-I inflation, and refitting the estimator on more observations brings Type-I error towards the nominal level with similar power.
  • Weak Overlap (Appendix D.15). With the default propensity clipping, weaker overlap reduces power while Type-I error stays at or below the nominal level.
  • Covariate-Dependent Image Outcomes (Table 1, Appendix C.2). In the medical-image experiments, XX now affects YY under both hypotheses. DR-KTE-Deep remains the method with the highest power, with Type-I error close to the nominal level.

We thank you again for engaging with us and for your valuable feedback. We hope these updates and our individual responses address your concerns. We would appreciate if you could take them into account in your final assessment, as we believe our representation-learning method for causal testing aligns well with ICLR.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.