acceptodds
Under review as a conference paper at ICLR 2027

Policy Learning under Stable Rank Dependence

Abstract

Policy learning aims to personalize treatment decisions to improve population welfare. Yet population welfare maximization does not account for individual-level benefit and harm. Such quantities depend on the joint distribution of potential outcomes, while observed data generally identify only their conditional marginals. Existing approaches either optimize over separate worst-case couplings across covariate values or impose parametric restrictions on the unidentified dependence between potential outcomes, leading to conservativeness or strong structural assumptions. In this paper, we study policy learning when rank dependence is stable across covariates, allowing the conditional marginals to vary freely while leaving the common copula otherwise unrestricted. This restriction forces the least-favorable copula to be chosen jointly over the covariate distribution, yielding a sharper maximin policy value than rectangular partial identification. We derive the sharp value, characterize exactly when copula invariance changes the optimal policy, and develop a finite-sample learner that directly optimizes the resulting objective without estimating the unidentified copula. We establish uniform policy-value and regret guarantees and quantify robustness to approximate violations of copula invariance. Experiments confirm the predicted policy-switch behavior and show that the learner remains effective under moderate violations of copula invariance.

Primary area:
causal reasoning
Keywords:
Causal Inference, Policy Learning, Individual Treatment Rule

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_e1d5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors offer an invariance assumption on the copula of the ranks of potential outcomes with and without treatment. Then, they provide a way to optimize an utility function when such an assumption holds using standard techniques from OT and optimization.

Critical strengths

Finds a theoretically sound simplification of an otherwise difficult problem. Full optimal policy learning with no structure on the conditional copula is a difficult objective and in practice would almost certainly result in valid bounds but which are too loose to be useful. The authors propose a simplification that on the theoretical side seems to provide necessary structure for a simple optimization for the policy. Furthermore, this simplification allows for possible sensitivity analysis under violations and the authors provide a method for robust learning.

Critical weaknesses
  1. It is unclear how much we have to pay for the robustness. The authors present us with a graph that shows how robust learning performs under correct specification of the violation parameter. However, it is unclear whether assuming a higher level of violation than the correctly specified one would produce a result less satisfactory than the competitors’ benchmark results. Methods of looking at violations are very good while you are in a vicinity of 0, yet deteriorate quickly to the performance of the worst-case policy learner otherwise. The authors do not convince me that this is not the case in this setting.

  2. The paper lacks motivation in existing literature and real-life applications. It is possible to come up with a number of simplifications to solve an otherwise too general optimization problem. However, these simplifications need to be grounded in some intuition, literature expertise from the same (or other) fields, or at least in empirical verifications and calculated metrics on true data. There seems to be little justification for the shared copula other than the possibility to do some nice, yet simple and straightforward math around it.

Questions for authors
  1. What would be some real-life applications where we can expect to have a shared-copula or small violations from it (small as measured in Figure 2)?

  2. How can the violation radius be estimated from data?

  3. Why are we using the continuous improvement score in the utility function? Is there any literature to back it up to be significant? Do practitioners care about it, or does it pose some nice theoretical properties that were otherwise not mentioned in the paper?

1 comment

Dear Reviewer e1d5,

Thank you so much for your followup. We want to express our deep appreciation for your constructive comments again, and will try our best to revise the paper according to your reviews. These suggestions are valuable to us for future submission! Wish you all the best!

Best regards,

Authors of paper 27621

Reviewer_wofMOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the policy learning problem when the dependency between the potential outcomes are unidentified. To this end, the paper assumes the copula describing dependence between potential outcomes is invariant across observed covariates. It also proposed a sharp maximin policy learning that achieves the usual O(n)O(\sqrt{n}) sample complexity rate.

Critical strengths
  1. The paper introduces the novel addition of the shared copula for the problem studied.
  2. The proposed algorithm seems practical and computationally tractable.
Critical weaknesses
  1. The paper has strong assumptions. In particular, it requires the dependence structure between potential outcomes to remain unchanged across covariate values, which is hard to justify in reality.
  2. In the motivation, the paper criticizes that worst-case joint distribution produces unnecessarily conservative policies. The paper itself also adopts a worst-case construction in~(2). Why does the worst-case copula policy would be less conservative than its worst-case joint distribution policy.
  3. All empirical results are reported in the shared-copula robust policy value. As all other benchmarks aim to optimize expected outcomes, it is no surprise that the proposed algorithm performs the best. Hence, the comparison is more informative if authors also report other statistics of the policy performance.
  4. Assumption 4 and LQL_Q are not defined before Corollary 1. The presentation of the paper should be modified. Furthermore, the bound of Corollary 1 is loose depending on the size of LQ,κL_Q,\kappa, which suggests that Corollary 1 may not necessarily explain the sensitivity results at the reported radii ρ\rho in Definition 3.
Questions for authors
  1. How are the results in Figure 2 implemented? In particular, how are the Vρ(π)V^{\rho}(\pi) and its oracle policy computed?
  2. Can the framework extend to discrete outcomes and multiple treatments?
  3. The writing of the paper needs further improvement. Currently, a lot of the important assumptions and definitions are not introduced before discussions on them, e.g. Assumption 4, regrets.
1 comment

Dear Reviewer wofM,

Thank you so much for your followup. We want to express our deep appreciation for your constructive comments again, and will try our best to revise the paper according to your reviews. These suggestions are valuable to us for future submission! Wish you all the best!

Best regards,

Authors of paper 27621

Reviewer_XPgsOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies policy learning when the goal is individual-level benefit, roughly the (smoothed) probability that the treatment a person receives beats the alternative by more than δ\delta. The role of this utility is somewhat confusing to me, since it motivates the whole paper while the later theory treats cπ,xc_{\pi,x} as a generic bounded cost. Because the utility compares both potential outcomes of the same person, it depends on their joint distribution, which the data cannot identify. Existing work deals with this either by taking a separate worst case at each xx or by assuming a parametric copula. The authors take a middle route and assume that the copula of the potential-outcome ranks is the same for all xx but otherwise unrestricted, an assumption that I find weakly justified and that cannot be checked from data. Under it, nature has to commit to one coupling for everyone, so the worst case becomes a single optimal transport problem whose value is never below the per-xx worst case. Building on this, the paper studies when the gap changes the optimal policy and proposes a learner that never estimates the copula, combining quantile estimation, Sinkhorn on a rank grid and gradient ascent on the policy. It also provides finite-sample regret bounds, a sensitivity bound for approximate invariance, and semi-synthetic experiments on ACIC 2016 data.

Critical strengths
  1. The paper provides a finite-sample analysis. It's good that the paper goes beyond identification and gives uniform value and regret guarantees for the shared-copula objective, without having to estimate the unidentified copula. The proofs in Appendix A appear correct, and the chain from Lemmas 2–6 to Theorem 2 is easy to follow, with each error source (sampling, quantile estimation, rank discretization, entropic regularization) getting its own term. Although the tools are standard, but they are put together carefully.
  2. The headline empirical effect is real and large where the assumption holds. In Table 1 the gain in true utility over the best baseline is +4.2 to +7.2 points out of ~50 (10 repetitions, SEs ≤ 0.1), increasing with the designed tail-shape parameter g as intended.
Critical weaknesses
  1. The shared-copula assumption is not well motivated. Everything in the paper rests on the copula being the same across xx, but the justification is a single sentence on p. 2 citing work on the conditional-copula "simplifying assumption". Those papers deal with pairs that are actually observed together, where the assumption can at least be checked. Here Y(0)Y(0) and Y(1)Y(1) are never seen together, so there is no way to test it. I would have liked to see a concrete application where it is plausible, or some discussion of when richer covariates make it more believable. My worry is that it fails exactly where personalization matters most, for example when an unmeasured factor drives treatment response in some subgroups but not in others. The paper also gives no guidance on choosing ρ\rho in Cor. 1, and the limitations section doesn't mention the assumption at all.
  2. Beyond the modeling choice, the novelty is limited. As I read it, the main new idea is the modeling step. Once the copula is shared, the worst case moves outside the expectation and becomes a single OT problem, and the method can be trained without estimating the copula. But this is also the part that is weakly motivated (see above). The rest of the theory is mostly standard. Prop. 1 essentially swaps the inf and the expectation, Lemma 1 restates the definition of HH, Prop. 2 is Birkhoff–von Neumann, and Lemmas 2, 3 and 6, Thm 2 and Cor. 1 follow from textbook OT and empirical-process arguments. Lemmas 4 and 5 are nice small observations, though elementary, and the smoothness that Lemma 5 avoids comes back in Cor. 1 through Assumption 4. More importantly, the theory never uses the structure of the benefit utility. Everything would go through for a plain mean-outcome objective, where the copula plays no role at all, so the theory says nothing about when the shared-copula restriction actually helps.
  3. The regret guarantee does not cover the algorithm that is actually run. Theorem 2 is proved for a finite policy class and assumes the inner optimization is solved well, while the implemented learner trains a neural policy by gradient ascent on a nonconvex objective. The policy class is never specified and the optimization error is never discussed. The discretization term is also loose at the experimental settings (see the Prop. 2 point below). And the quantity one really cares about, the probability of benefit, corresponds to κ→0\kappa \to 0, where the bounds blow up because of the 1/κ1/\kappa factor. κ\kappa is never varied in the experiments either.
  4. The experiments are limited. All experiments use a single synthetic design in which the shared-copula assumption holds exactly, so they mostly show that the method works when its own assumption is true. The robustness study in Sec. 5.3 never generates data that violate the assumption and only changes how policies are evaluated, and the text describes Fig. 2 more favorably than the bars suggest. Much of the evidence is also measured by regret on the paper's own objective, which favors the method by design. The baselines are weak for this problem, since the mean-based learners cannot respond to the design, and benefit-aware or parametric-copula methods are missing. There are no ablations over the key tuning choices (in particular κ\kappa), no real outcome data, and no reporting of benefit or harm rates even though harm motivates the paper. Finally, important implementation details are missing and no code is provided, which makes the results hard to reproduce.
  5. Theorem 1 does not add much. By Lemma 1, Vcc=Vrect+HV^{cc} = V^{\text{rect}} + H, and condition (6) is just this identity rearranged. So the theorem says πR\pi_R is suboptimal under VccV^{cc} exactly when some other policy has a higher VccV^{cc}, which is true by definition, and the proof in A.4 is a two-line substitution. The abstract presents this as characterizing exactly when copula invariance changes the optimal policy, but the condition is written in terms of HH and VrectV^{\text{rect}} themselves and says nothing about the marginals, covariates or utility. The split into a gain term and a deficit term is a useful remark, and I think it would fit better right after Lemma 1. The experiments also never report HH or the gap in (6), so the claimed policy-switch behavior is never checked against the theorem.
  6. Prop. 2 relies on an assumption that rarely holds. Prop. 2 asks the cost cπ,xc_{\pi,x} to be constant on each cell of a rank grid, and I don't see how this can hold in the paper's own setting. With continuous marginals the cost changes smoothly within each cell, so the condition is only met in trivial cases, for example when treatment is clearly better or clearly worse for everyone at a given xx. So Prop. 2 is really a statement about the discretized problem that the algorithm solves. That is fine in itself, but the paper presents it as if it solved the continuous problem. The connection back to the continuous problem goes through Lemma 5, and that bound is quite loose at the settings used in the experiments. I'd suggest stating Prop. 2 directly for the discretized objective, or explaining why the cell-constancy assumption is reasonable.
Questions for authors
  1. In Table 1, did the OR/IPW/DR and rectangular baselines use the true nuisance functions or estimated ones? If they used estimates while your learner used the true quantiles, how do the numbers change when everyone gets the true nuisances?
  2. Could you add an experiment where the copula actually varies with xx, for example a Gaussian copula whose correlation differs across strata, and report VtrueV^{\text{true}} for all methods as the violation grows? This is what the abstract claims to show, and it would change my assessment.
  3. It would also help to see a shared copula other than independence (comonotone, countermonotone, or Gaussian with strong correlation), as well as a setting where H(π)≈0H(\pi) \approx 0, to check that the method does no harm when the restriction doesn't help.
  4. How are Vρ(π)V_\rho(\pi) and the oracle πρ⋆\pi^\star_\rho computed in Sec. 5.3? And why does your method's regret in Fig. 2 go down and then back up as ρ\rho grows?
  5. Is Prop. 2 meant as a statement about the discretized problem? If so, please say so, and clarify what "directly computable" means for the continuous H(π)H(\pi).
  6. What policy class do you use in Alg. 1, and how do you control the optimization error ξn\xi_n? Since V^M,ε\widehat V_{M,\varepsilon} is a minimum of functions that are affine in (π(X1),…,π(Xn))(\pi(X_1), \dots, \pi(X_n)), it is concave in π\pi, and pointing this out would help close part of the gap between the theory and the algorithm.
  7. To support the policy-switch claim, could you report the quantities from Theorem 1 in Sec. 5.1, i.e. H(π^cc)H(\hat\pi_{cc}), H(π^rect)H(\hat\pi_{\text{rect}}), the gap in (6), and how often the two learned policies disagree on the test set?
  8. How does this work relate to Ji, Lei & Spector (2023, arXiv:2310.08115) on covariate-assisted bounds, and to copula-stability assumptions in panel data (Callaway & Li, 2019)? Also, is the "rectangular" baseline your own re-implementation for the sδ,κs_{\delta,\kappa} utility? Cui & Han (2025) target a different welfare criterion, so this should be stated clearly.
1 comment

Dear Reviewer XPgs,

Thank you so much for your followup. We want to express our deep appreciation for your constructive comments again, and will try our best to revise the paper according to your reviews. These suggestions are valuable to us for future submission! Wish you all the best!

Best regards,

Authors of paper 27621

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.