- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper develops two complementary methods for uniform inference in longitudinal randomized studies: a covariance-robust confidence band for the population treatment-effect surface and a conformal prediction band for a new subject’s latent treatment-effect curve. The population method combines working-independence regression, subject-block CR3 covariance estimation, and Gaussian multiplier calibration, while the personalized method combines FPCA, history-based score prediction, and recovery-set conformal calibration. Simulations and a HeartSteps analysis suggest that the population band approaches nominal coverage as sample size increases, whereas the protected personalized bands achieve conservative coverage but can be substantially wider.
Critical strengths
None
Critical weaknesses
Apparently, one of the paper’s many weaknesses is its logical flow.
-
Setting aside technical correctness and novelty, I have substantial concerns about the manuscript's readability and practical framing, even as a researcher with expertise in healthcare and clinical trials. The introduction opens with "Repeatedly randomized longitudinal studies contain information at two levels," without first explaining the study design or providing a concrete motivating example. The term "repeatedly randomized longitudinal studies" should be defined and its relationship to the micro-randomized trial setting used in the application clarified. Conference page limits do not justify omitting the context readers need to understand the problem; additional background could also be provided in the supplementary material.
-
Given the limited introductory context, I turned to Section 4.4, "HeartSteps Population Analysis," to understand the practical motivation. However, this section again begins with the number of participants and decision points, followed immediately by numerical results, rather than clearly stating the substantive research question. Before presenting these results, the authors should explain what they are trying to learn from HeartSteps and how the findings should be interpreted in the application's context.
-
The supplementary HeartSteps analysis provides additional design and implementation details, along with some visualization, but it still does not adequately connect these elements to a clearly articulated practical problem. A reader-oriented explanation would help establish connection.
-
Overall, my judgement is that, the manuscript has a substantial problem with narrative flow: it prioritizes technical constructions and numerical results before establishing a compelling practical motivation. Readers need to understand what problem is being solved, why it matters, and what existing methods fail to provide before they can meaningfully assess the proposed contribution.
-
Technical rigor remains essential, but extensive derivations and proofs, whether developed with or without AI assistance, cannot substitute for a clearly motivated research question and logical writing. In its current form, the manuscript has not adequately communicated the practical significance of its contribution, and this is a substantive concern rather than a minor stylistic issue.
Questions for authors
My current questions concern the scientific motivation, data selection, and practical interpretation of the application. Clarifying these points would help me assess the contribution and formulate any further technical questions.
-
What specific scientific question does the HeartSteps reanalysis answer, and how does it extend the question addressed by the original study?
-
Starting from the public HeartSteps V1 data, please provide or identify a participant- and decision-level exclusion flow leading to the 37 participants and 5,777 decision points. In particular, what is the scientific justification for requiring observed outcomes and consistent randomization and delivery indicators? Do any exclusions depend on information obtained after treatment assignment, and how might this selection affect the population represented by the analysis and the interpretation of the fitted contrasts?
-
Were the scientific framing and analysis choices evaluated with researchers experienced in micro-randomized trials, behavioral interventions, or clinical-trial statistics? Without identifying individuals, please describe any substantive input and how it informed the pooling of active and anti-sedentary suggestions, the evaluation grid, and the interpretation of the results. If no such consultation occurred, what study-specific evidence supports these choices? Or, is this purely academic research purpose?
-
It seems that the HeartSteps application demonstrates only the population layer. What concrete real-study setting motivates the personalized prediction layer, and what evidence supports its feasibility there? Please explain how calibration subjects would supply the required disjoint history and recovery observations, and how investigators would assess whether the resulting bands are sufficiently informative for the intended scientific question. Please distinguish demonstrated feasibility from a proposed future application.