- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies enrollment allocation across multi-site randomized clinical trials when all of these sites differ in target-population characteristics. The paper developed a privacy-aware framework for estimating a target ATE that is for a target-population generally differs from the trial populations. This is a work with great potential, but the current shape is not ready for a publication.
Critical strengths
The paper's theoretical investigation is quite extensive and in a good depth, despite they can be shaped and organized better for presentation. The main text should be more readable.
Critical weaknesses
-
The paper needs substantial language polishing before going to peer review. Many terms are very vague when they are first introduced. For example: "...To answer this question, we study multisite randomized trials with a common linear treatment effect, known site covariate second moments and target mean, and site-specific participant-level zero-concentrated differential privacy (zCDP) constraints..." Here "site covariate second moments and target mean" is hard to understand.
-
The motivation for this problem is weak or not stated clearly. I understand that the authors wanted to say the target-population is different to the underlying "super-population" targeted by the trial participants. However, readers will still be confused when they didn't read the paper in details.
For example, when I first read the abstract, I don't understand why these are required in estimating a target ATE: "...requires allocating limited enrollment across sites that differ in target population relevance, recruitment costs, and privacy constraints." If I assume the trial participants can represent the target population, then under the site-specific complete randomization, one can just use the target-site data to estimate it, which doesn't require any external information.
Even now I understand the trial population is different, I still don't see why some of these factors such as the privacy constraint are requirements. I feel the author(s) might want to say this is what observed in their specific setting or motivating data example, rather than a broadly or generally existed data structure or the rule of such settings.
Therefore, I encourage the authors to substantially revise their framing clearly. For abstract, I recommend to keep in concise and avoid specifying too dense information.
- The literature review is weak. It is unclear why the proposed method is unique or in what sense it is more advantageous to other existing works. For example, there are some connections to transportability and federated/transfer learning papers should be highlighted. Federated and transfer learning are also specific for tackling privacy concerns. Example papers include, but are not limited to:
Lee, Dasom, et al. "Improving trial generalizability using observational studies." Biometrics 79.2 (2023): 1213-1225.
Han, Larry, et al. "Federated adaptive causal estimation (face) of target treatment effects." Journal of the American Statistical Association 120.551 (2025): 1503-1516.
Colnet, Bénédicte, et al. "Causal inference methods for combining randomized trials and observational studies: a review." Statistical science: a review journal of the Institute of Mathematical Statistics 39.1 (2024): 165.
Some of these papers may assume some cross-sites conditional homogeneity assumptions that cannot be used in the current setting, but if the authors can highlight them, I think it will strengthen the paper.
- Figure 1 is useful, but the term "Privacy-protected" in the orange boxes needs some explanation. Do you mean even within each site, there are privacy concerns, or only when these information are shared to the analysis center, there is such a concern.
In what level the privacy is protected? The summary statistics are participant-level or site-level crude summaries. The site-level summaries are, for example, the mean of ages in a site. The participant-level summaries can be, for example, a summary score of each participant's covariates, without needing to know their specific age, gender and ID information after summarizing, then the shared information.
-
The randomization is characterized by 𝐴 |𝑋,𝐽= 𝑠∼Bernoulli(1/2). This is strange to me. I think it should be 𝐴∼Bernoulli(1/2) or 𝐴 | 𝐽= 𝑠∼Bernoulli(1/2). Conditioning on covariates may not result in a marginally 1:1 randomization.
-
Y(0) and Y(1) are not defined. If you work under the potential outcomes framework, you should at least cite the original Neyman and Rubin paper and briefly state the potential outcomes notion. It would also be helpful to separate underlying full data and observed data structure clearly. Missing stating some identification assumptions, such as positivity and correct model specifications...
-
Equation (1) is confusing, and not sure why that model is assumed. This is a very strong and maybe not testable assumption for the outcome model. Why can't the framework accommodates the even simpler linear regression or for all generalized/general linear models, or why doesn't the framework allow a more general, unknown Y(a) ~X structure non-parametrically? Is this outcome structure common in the specific setting the author(s) considered and already with clear justification for the specific model form used?
-
Many information are extremely hard to follow in the main text. Missing some brief review or introduction to zCDP method as this is a component or assumption made in the author(s)' method. The feeling of the overall reading of this paper is that there is not a key focus and each piece of the presentation is not organized well. Consider moving more technical details to the appendix.
-
When reading the entire Section 5, most content focused on mathematical techniques, which is good and I appreciate the author(s)' efforts on showing these math, but there should be at least some illustrations, remarks or English summary on which step(s) resolve which concerns in the motivating example. For example, what model or problem formulation helps resolving privacy concern in the level the author(s) wanted to address. The main text should try to achieve a better balance on "technical-depth-intuition trade-off".
Questions for authors
None.