- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies whether deterministic completion accuracy, measured by aggregate missing-region RMSE, is sufficient to characterize the usefulness of a completion as a condition for a downstream stochastic time-series generator. The authors compare two interfaces: Full, which passes the dense completion directly to the generator, and Avg16, which applies a width-16 temporal moving average. They construct residual interventions by temporally shuffling completion residuals, reapplying the observation mask, and rescaling the residuals to match the original pooled missing-region RMS. The downstream effect is measured through the change in the Full-versus-Avg16 CRPS gap.
Critical strengths
The paper identifies a real limitation of evaluating an intermediate completion only through pointwise reconstruction metrics. In a completion-to-generation pipeline, a completion may be close to the target while still being difficult or misleading for a finite downstream model to use.
Critical weaknesses
The paper structure is disorganized. For example, the motivation, problem formulation, and organization of evidence are not sufficiently connected in intro. After briefly introducing the completion-to-generation setting, the introduction moves directly to the Full/Avg16 interfaces and their comparison statistics. The discussion of how the closest prior work handles conditioning, what its evaluations leave unresolved, and how the proposed comparisons address that gap appears only afterward and remains brief. Consequently, when these definitions are introduced, readers have not yet been given sufficient motivation for why comparing dense and smoothed conditions is an appropriate way to answer the research question.
The paper also needs to distinguish more clearly among three claims: aggregate RMSE is insufficient to characterize downstream utility; the temporal assignment of residuals affects interface utility; and the direction of this effect depends on the domain. These claims require different levels of experimental control. In particular, the in-mask permutation in Appendix A.2.5 preserves MAE and per-window RMSE, and the negative mean shifts observed on Stock under the primary intervention become positive under this stricter control. This result materially affects the interpretation of the prominently discussed Stock reversal, yet it is not adequately integrated into the main argument. I recommend establishing the prior findings and specific research gap before introducing the experimental statistics, explaining how the formulation and interventions address that gap, and organizing the results around the evidence supporting each claim.
Questions for authors
- What is the primary claim of the paper?
- Why is the in-mask control not included in the main paper, given that it materially changes the Stock result?