acceptodds
Under review as a conference paper at ICLR 2027

Structurally Compatible Generative Ambiguity Sets via Exogenous-Space Factorization

Abstract

Distributionally robust optimization (DRO) critically depends on the ambiguity set used to characterize plausible distribution shifts. Recent generative approaches parameterize adversarial distributions with trainable generative models, enabling flexible shifts beyond the empirical support, but they do not explicitly preserve known causal structure. In contrast, structural causal optimal transport restricts admissible distributions according to a structural causal model (SCM), but is not naturally formulated as a trainable generative adversary. We bridge these perspectives with exoDRO, which constructs generative ambiguity sets in the exogenous space of a bijective SCM. Rather than perturbing the endogenous distribution directly, we independently parameterize the exogenous marginals and push their product distribution through the fixed structural map. This construction guarantees structural compatibility for every adversarial parameter value by design. We further show that the inclusive KL divergence between the nominal and adversarial endogenous distributions decomposes exactly into the sum of the corresponding exogenous marginal KL divergences. This identity holds for any bimeasurable bijective structural map and does not depend on causal depth, Jacobian volume preservation, or additive-noise structure. Combined with likelihood-based generative consistency bounds, this identity yields a global reconstruction-based certificate and finite-sample control using a single dataset shared across all exogenous coordinates. To optimize the resulting coupled adversarial problem, we develop a cyclic block-coordinate scheme whose fixed-block subproblems reduce to single-model generative-DRO inner problems. Experiments verify the predicted structural properties and show that, on the predictive benchmark, exoDRO achieves the lowest prediction error across the evaluated nominal and held-out OOD settings compared with classical, generative, and structure-aware DRO baselines.

Primary area:
optimization
Keywords:
Distributionally Robust Optimization, Generative Ambiguity Sets, Structural Causal Models, Causal Optimal Transport

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_SgdPOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper enhances DRO with a structural causal model (SCM). It constructs generative ambiguity sets in the exogenous space of a bijective SCM. Rather than perturbing the endogenous distribution directly, we independently parameterize the exogenous marginals and push their product distribution through the fixed structural map. Experiments verify the predicted structural properties and logical claim of this study.

Critical strengths
  1. Well-motivated problem. Generative DRO does not guarantee structural compatibility, while SCOT is not trainable — both gaps are real, and bridging them is a worthwhile research direction.

  2. Elegant parameterization. Placing the adversary in the exogenous space makes structural compatibility a byproduct of the construction rather than a constraint — this "by design" observation is tasteful in itself.

  3. Direct empirical verification of the theory. The exact identity is verified numerically across 20 configurations, with an additional cross-check using an independent estimator.

Critical weaknesses
  1. The core "theorem" is theoretically insufficient, and its impact on machine learning research needs to be articulated: how does this paper differ from "Designing Ambiguity Sets for Distributionally Robust Optimization Using Structural Causal Optimal Transport"?

  2. There is a lack of discussion on optimization cost and scalability. Please provide the analysis on the complexity.

  3. There is no sensitivity analysis to misspecification / estimation error of gg.

  4. All methods' MSEs are compressed in the narrow band 0.090–0.103. The gaps between exoDRO and SCOT — ID 0.0904 vs 0.0906, mild 0.0916 vs 0.0917, moderate 0.0914 vs 0.0918 — fall in the range 0.0002–0.0004, which completely overlaps with the standard deviations across 5 seeds (±0.0002–0.0005). Claiming "the lowest across all evaluated settings" without any significance testing is not acceptable. Even the 1.7% relative improvement at strong OOD (0.0909 vs 0.0925) requires support from paired tests.

  5. The experiments use only synthetic datasets, with no experiments on mainstream datasets. Please conduct experiments on datasets such as CIFAR.

  6. The presentation of results throughout the paper is monotonous. The authors should follow "Designing Ambiguity Sets for Distributionally Robust Optimization Using Structural Causal Optimal Transport" and perform trajectory visualization experiments.

Questions for authors
  1. Sensitivity to a misspecified / approximately invertible structural map. Your Theorem 1 relies on the structural map gg being a known bimeasurable bijection. When gg must instead be estimated from data and is only approximately invertible, what form does the exact KL decomposition degrade into? Can you characterize the order of magnitude of the resulting error term (e.g., in terms of the estimation error of gg or of its inverse), even approximately?

  2. Statistical significance of the predictive results. Could you provide paired significance tests for exoDRO vs. SCOT across the evaluated settings — for example, a Wilcoxon signed-rank test across training seeds, or bootstrap confidence intervals for the mean MSE differences? Since the reported improvements over SCOT are small in absolute terms (e.g., 0.09160.0916 vs. 0.09170.0917 at the mild OOD level, against seed standard deviations of ±0.0002\pm 0.0002–0.00050.0005), significance tests are needed to support the claim of being lowest across all settings.

  3. Computational cost. What is the total training cost of exoDRO (wall-clock time and number of models) relative to the single-model GAS-DRO baseline?

  4. Selected robustness budgets. Could you report the final robustness budgets selected for each method via the OOD-validation procedure — the Wasserstein radius for W-DRO, the KL radius for KL-DRO, the transport radius for SCOT, the ambiguity budget for GAS-DRO, and the coordinatewise budgets ϵi\epsilon_i for exoDRO? Since these geometries are not directly comparable, knowing the operating points is important for interpreting the comparison.

Reviewer_kn3bOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes exoDRO, a generative distributionally robust optimization framework for a known bijective structural causal model with independent exogenous variables. It parameterizes the exogenous marginals separately and pushes their product distribution through the fixed structural map, ensuring structural compatibility throughout optimization. The paper establishes an exact decomposition of endogenous inclusive KL divergence into marginal exogenous KL divergences, derives reconstruction-based and finite-sample bounds, and develops a cyclic block-coordinate optimization procedure. Experiments on synthetic regression and controlled structural examples demonstrate the expected structural properties and improved predictive performance over the evaluated DRO baselines.

Critical strengths
  1. The parameterization provides a useful way to enforce structural compatibility. Independent exogenous generators followed by the fixed structural map ensure compatibility at every optimization iterate. This avoids an additional structural penalty or projection step. The resulting separation between marginal distributional constraints and the coupled downstream objective is clear and useful for constructing structured generative adversaries.

  2. The controlled evaluation provides encouraging evidence within the stated setting. The predictive comparison includes both generative and structure-aware baselines, uses separate validation and test environments, and reports results over five training seeds. The environment-matched excess-risk analysis is particularly helpful given the substantial irreducible noise. For example, under strong shifts, the reported excess MSE decreases from 0.0028 for SCOT to 0.0013 for exoDRO. The paper also clearly states the limitations of its assumptions and optimization guarantee.

Critical weaknesses
  1. The additional contribution beyond the existing ingredients appears limited. Theorem 1 follows directly from KL invariance under a bimeasurable bijection and KL additivity over product measures. Structural compatibility follows from the adopted definition and parameterization, while the finite-sample analysis uses standard uniform-convergence and union-bound arguments. The optimization guarantee is conditional and inherited from the single-model procedure of Wen and Yang. These observations form a coherent framework, but I am not yet convinced that the resulting theoretical advance is substantial. Moreover, the experiments set the marginal constraint directly to KL with (c_i=0), so they do not demonstrate the practical value or calibration of the reconstruction-based certificate emphasized in the paper.

  2. The empirical evidence does not sufficiently establish the source or breadth of the predictive gains. The predictive results come from a single four-feature synthetic regression benchmark with an exactly known SCM and Gaussian exogenous mean/scale shifts. The structural diagnostics use a three-variable linear additive chain. Within this setting, the paper does not include a non-adversarial SCM-based generation control with the same sampling budget and label-generation procedure. Such a comparison is important for determining how much improvement comes from the proposed adversarial optimization rather than from access to additional structurally consistent training samples. The narrow evaluation and missing attribution control leave the practical significance of the method insufficiently established.

Questions for authors
  1. Clarification of label generation and information access. How are training labels obtained for newly generated feature vectors in the predictive experiment? Is the ground-truth target mechanism in Appendix E.1 queried during training? Please specify which methods have access to this mechanism and how generated features and labels are handled by each relevant baseline. This information is necessary to interpret the predictive comparison.

  2. Focused additional evidence on the source of improvement. Within the existing benchmark, how does exoDRO compare with non-adversarial SCM-based sampling using the same label-generation procedure and sampling budget? This control would materially affect my assessment of whether the gains require the proposed adversarial ambiguity construction.

  3. Clarification of the theory–implementation connection. Appendix E.1 uses (J_i=D_{\mathrm{KL}}(P_{U_i}\Vert P_{\theta_i})). Is this quantity estimated solely from the nominal training observations, or using additional samples from the known exogenous distributions? Which assumptions of the finite-sample analysis, particularly the uniformly bounded per-sample loss assumption, hold for this implementation? Please distinguish the guarantees applicable to the implemented method from those established for the general reconstruction-based formulation.

Reviewer_EsKaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes exoDRO, which builds robust ambiguity sets by learning independent distributions over the noise variables of a known, invertible structural causal model. This preserves structural compatibility and gives an exact decomposition of the observed-space KL divergence into marginal terms. The paper also provides statistical bounds, a blockwise optimization scheme, and synthetic experiments.

Critical strengths
  • Enforcing independence through the model architecture is a clean idea. Structural compatibility holds throughout optimization, without relying on a dependence penalty.
  • The KL decomposition gives the coordinatewise budgets a clear interpretation. I also appreciate the explicit discussion of assumptions and the limits of the optimization guarantee.
  • The reported results improve on SCOT across the evaluated settings. Using five training seeds, separate validation environments, and the excess-risk analysis makes the empirical comparison more informative.
Critical weaknesses
  • The theoretical advance seems modest. The main identity follows directly from KL invariance under a bijection and additivity for product distributions; the statistical and fixed-block optimization results largely assemble existing tools.
  • Predictive evaluation is limited to one small synthetic benchmark with four input variables and an exactly known structural model. It leaves practical sensitivity to model estimation error and scalability unresolved.
  • The structural diagnostic uses a joint adversary with an explicitly correlated base distribution, so dependence is built into that comparison. A predictive ablation with matched model capacity and training effort would better isolate the benefit of factorization.
Questions for authors
  • How quickly does performance deteriorate when the structural coefficients or inverse map are slightly misspecified?
  • Does the predictive advantage remain when factorized and joint adversaries use comparable capacity, computation, and realized KL budgets?
  • Since the evaluation shifts are Gaussian, how much is gained over simply optimizing independent Gaussian exogenous distributions?
  • What are the runtime and training-stability costs as the number of coordinates and inner updates increase, given that the convergence result covers only a fixed block?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.