- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how external control data can improve estimation precision in randomized trials when the two data sources have different distributions. The method adjusts for differences in covariates and conditional outcomes. It also uses adaptive shrinkage to rely more on trial data when the models are misspecified, under certain conditions. The paper provides consistency and asymptotic efficiency analyses, supported by simulations and a real-data application.
Critical strengths
-
Trial participants and external controls often differ in practice. The method directly accounts for these differences, which gives the work a clear motivation.
-
The method combines distribution calibration with adaptive shrinkage. It also uses covariate information from external controls to estimate the mean outcome in both trial arms more precisely. This is a useful contribution.
-
The theory covers consistency, efficiency gains, and uncertainty from estimating distribution shifts. The authors also explain the limits of the method under local model misspecification.
Critical weaknesses
-
The gains in precision rely on reliable information about how the trial and external populations differ. Although several possible sources are discussed, I am not sure how confidently these differences can be modeled in practice, especially for outcomes. It is also unclear when these assumptions are more reasonable than exchangeability.
-
It is unclear to me when the method offers enough practical benefit to justify the extra modeling and validation data. Shrinkage can protect the estimate by falling back on trial data, but this may also leave little gain from the external controls. The settings where the method is most useful remain unclear.
-
The real-data example includes six covariates, but the simulations use only one. This leaves me unsure how the method performs when several covariates differ between the two populations. The real-data results alone cannot establish estimation accuracy because the true treatment effect is unknown.
Questions for authors
-
What makes the assumptions about population differences credible in a real application? When would they be easier to justify than exchangeability?
-
In which practical settings are the gains likely to justify the extra modeling and validation data? When would using trial data alone be preferable?
-
How does the method perform when several covariates differ between the two populations? Do the gains in precision and the confidence interval coverage remain similar?