acceptodds
Under review as a conference paper at ICLR 2027

Experimental Design for Target-Population Treatment Effect Estimation under Heterogeneous Privacy Constraints

Abstract

Estimating a target-population average treatment effect (ATE) in multisite randomized trials requires allocating limited enrollment across sites that differ in target-population relevance, recruitment costs, and privacy constraints. Existing target-population trial designs do not directly account for site-specific privacy constraints, while private multisite ATE estimation generally takes site enrollment as given. In this paper, we study experimental design for multisite randomized trials under site-specific participant-level privacy. We characterize each site's privacy-constrained information contribution and derive a target-specific criterion that balances estimation variance and worst-case squared bias. The criterion characterizes the minimax mean squared error within constants independent of the budget, site count, and allocation. A site's information contribution grows quadratically with enrollment in the privacy-limited branch and linearly after a site-specific threshold, so heterogeneous privacy can change the allocation favored by classical design. The resulting design problem is strongly NP-hard even in the high-privacy regime. We develop sparse conic constructions with dimension-dependent approximation guarantees under explicit solver-accuracy conditions, budget-feasible integer recovery, and private estimator weights. Experiments show when accounting for heterogeneous privacy reduces target-population ATE error under a fixed enrollment budget and when the added privacy variance outweighs the reduction in squared bias.

Primary area:
causal reasoning
Keywords:
Causal Inference, Experimental Design, Privacy Constraints, Treatment Effect

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_jzSJOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies enrollment allocation across multi-site randomized clinical trials when all of these sites differ in target-population characteristics. The paper developed a privacy-aware framework for estimating a target ATE that is for a target-population generally differs from the trial populations. This is a work with great potential, but the current shape is not ready for a publication.

Critical strengths

The paper's theoretical investigation is quite extensive and in a good depth, despite they can be shaped and organized better for presentation. The main text should be more readable.

Critical weaknesses
  1. The paper needs substantial language polishing before going to peer review. Many terms are very vague when they are first introduced. For example: "...To answer this question, we study multisite randomized trials with a common linear treatment effect, known site covariate second moments and target mean, and site-specific participant-level zero-concentrated differential privacy (zCDP) constraints..." Here "site covariate second moments and target mean" is hard to understand.

  2. The motivation for this problem is weak or not stated clearly. I understand that the authors wanted to say the target-population is different to the underlying "super-population" targeted by the trial participants. However, readers will still be confused when they didn't read the paper in details.

For example, when I first read the abstract, I don't understand why these are required in estimating a target ATE: "...requires allocating limited enrollment across sites that differ in target population relevance, recruitment costs, and privacy constraints." If I assume the trial participants can represent the target population, then under the site-specific complete randomization, one can just use the target-site data to estimate it, which doesn't require any external information.

Even now I understand the trial population is different, I still don't see why some of these factors such as the privacy constraint are requirements. I feel the author(s) might want to say this is what observed in their specific setting or motivating data example, rather than a broadly or generally existed data structure or the rule of such settings.

Therefore, I encourage the authors to substantially revise their framing clearly. For abstract, I recommend to keep in concise and avoid specifying too dense information.

  1. The literature review is weak. It is unclear why the proposed method is unique or in what sense it is more advantageous to other existing works. For example, there are some connections to transportability and federated/transfer learning papers should be highlighted. Federated and transfer learning are also specific for tackling privacy concerns. Example papers include, but are not limited to:

Lee, Dasom, et al. "Improving trial generalizability using observational studies." Biometrics 79.2 (2023): 1213-1225.

Han, Larry, et al. "Federated adaptive causal estimation (face) of target treatment effects." Journal of the American Statistical Association 120.551 (2025): 1503-1516.

Colnet, Bénédicte, et al. "Causal inference methods for combining randomized trials and observational studies: a review." Statistical science: a review journal of the Institute of Mathematical Statistics 39.1 (2024): 165.

Some of these papers may assume some cross-sites conditional homogeneity assumptions that cannot be used in the current setting, but if the authors can highlight them, I think it will strengthen the paper.

  1. Figure 1 is useful, but the term "Privacy-protected" in the orange boxes needs some explanation. Do you mean even within each site, there are privacy concerns, or only when these information are shared to the analysis center, there is such a concern.

In what level the privacy is protected? The summary statistics are participant-level or site-level crude summaries. The site-level summaries are, for example, the mean of ages in a site. The participant-level summaries can be, for example, a summary score of each participant's covariates, without needing to know their specific age, gender and ID information after summarizing, then the shared information.

  1. The randomization is characterized by 𝐴 |𝑋,𝐽= 𝑠∼Bernoulli(1/2). This is strange to me. I think it should be 𝐴∼Bernoulli(1/2) or 𝐴 | 𝐽= 𝑠∼Bernoulli(1/2). Conditioning on covariates may not result in a marginally 1:1 randomization.

  2. Y(0) and Y(1) are not defined. If you work under the potential outcomes framework, you should at least cite the original Neyman and Rubin paper and briefly state the potential outcomes notion. It would also be helpful to separate underlying full data and observed data structure clearly. Missing stating some identification assumptions, such as positivity and correct model specifications...

  3. Equation (1) is confusing, and not sure why that model is assumed. This is a very strong and maybe not testable assumption for the outcome model. Why can't the framework accommodates the even simpler linear regression or for all generalized/general linear models, or why doesn't the framework allow a more general, unknown Y(a) ~X structure non-parametrically? Is this outcome structure common in the specific setting the author(s) considered and already with clear justification for the specific model form used?

  4. Many information are extremely hard to follow in the main text. Missing some brief review or introduction to zCDP method as this is a component or assumption made in the author(s)' method. The feeling of the overall reading of this paper is that there is not a key focus and each piece of the presentation is not organized well. Consider moving more technical details to the appendix.

  5. When reading the entire Section 5, most content focused on mathematical techniques, which is good and I appreciate the author(s)' efforts on showing these math, but there should be at least some illustrations, remarks or English summary on which step(s) resolve which concerns in the motivating example. For example, what model or problem formulation helps resolving privacy concern in the level the author(s) wanted to address. The main text should try to achieve a better balance on "technical-depth-intuition trade-off".

Questions for authors

None.

Reviewer_jsD1OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposed a experimental design for multisite randomized trial with different privacy constraint under parametric model assumption.

Critical strengths
  1. It is a very math-focusing paper with well-defined support and risk.
  2. The paper solves a quite relevant problem about targeted population ATE but under participant-level privacy constraints.
Critical weaknesses
  1. missing citations, like privacy-wise paper:https://arxiv.org/pdf/2505.21641 and https://proceedings.iclr.cc/paper_files/paper/2025/file/6baec7c4ba0a8734ccbd528a8090cb1f-Paper-Conference.pdf. And for adaptive experimentation: https://arxiv.org/abs/2403.03589
  2. According to the problem setup part, the whole paper is actually considering the parametric model, which I think is kind of the constrain of application in the real-world.
  3. The problem setup part especially the inputs paragraph should have more elaborated explanation of what is the w and r is. Now is too scattered and hard to read.
  4. The paper is really hard to understand and read with no strengthen and weird change lines between line 448 to 450.
Questions for authors
  1. Is the title format very weird and to wide. Need double check here.
  2. The paper
  3. Could the authors providing more illustration about how the privacy mechanism leading problem here? and also the author leading too many notations and could be explain more clearly.
  4. In the beginning of experiment section, you mentioned that there is three research questions. How does the whole experiment section answer this three questions?
Reviewer_gvubOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a privacy-preserving experimental design for multisite RCTs. Unlike conventional RCT analyses that pool data in one place, the proposed approach shares only local summary statistics, with local differential privacy applied according to each site’s privacy requirements. Theoretically, the authors characterize each site’s privacy-constrained information contribution and derive a target-specific criterion that balances estimation variance and worst-case squared bias. This shows that minimax mean squared error within constants independent of the budget, site count, and allocation. They also prove that design problem is strongly NP-hard and develop conic approximations, sparse site selection, and budget-feasible integer enrollment allocations. The numberical study examine after accounting for heterogeneous privacy requirements reduces target-population ATE estimation error under a fixed recruitment budget, and when the additional privacy variance outweighs the reduction in squared bias.

Critical strengths

The key strength of the proposed experimental design is its enrollment allocation framework for multisite RCTs under a fixed budget, rather than privacy in my mind. The algorithm considers recruitment costs and each site’s contribution to estimating the target-population ATE, making it relevant to practical trial planning. Instead of data centralization as classical RCT, the proposed method only need to share local summary statistics, which makes the framework fit in federated trials and meta analyses. The author also provide theorical guarantee: under the stated assumptions, the authors establish a constant-factor minimax risk characterization, prove the computational difficulty of global optimization, and provide approximation guarantees for their proposed algorithms.

Critical weaknesses

My main concern is whether the proposed privacy constraints are feasible with the implement of an RCT. Multisite RCTs generally share a common protocol, but also ethics, data-sharing, privacy, and security agreements governing access to individual data. I am not sure whether DP is redundant and whether its accuracy cost is justified.

Furthermore, RCTs is not just about ATE but also safety assessment. E.g., when a individual develops an (S)AE, individual data may need to be reported to the research team for investigation and decisions about treatment continuation. For higher-risk cases, a DSMB will be organized and require detailed safety information. If the research team receives only DP aggregate summaries, how would it identify unusual clinical measurements or outcomes requiring further investigation?

Further clinical investigation, such as after intervention follow up and subgroup analysis, may also be needed for particular participants, including those with extreme outcomes, allergic reactions, or pregnancies during the intervention. The proposed private summaries would not allow the research team to identify these participants.

Questions for authors

For RCTs, it is more common to determine the sample size first, then budget and recruitment. I am curious about how to do a power-based sample-size calculation, where only private local summaries are aggregated? Clearly, DP may require a larger sample size, but this can be difficult to account for at the design stage.

In my opinion, the upper bound in Theorem 2 is loose, particularly in its dependence on L, although this is fine to me. Could the authors discuss how the theoretical guarantee can guide practical enrollment allocation? Or this require a specific private estimator?

Site covariate moments and the target-population mean are typically not known exactly. How sensitive are the proposed allocations and ATE estimates to estimation errors or misspecification of these inputs?

Could the authors compare the proposed estimator with a pooled-data analysis for ATE estimation? This would help isolate the accuracy cost of privacy protection and sharing only local summaries.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.