- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper estimates unrecoverability and model-specific error to select reliable time-series imputations.
Critical strengths
The conditional Bayes-risk formulation clearly separates contextual unrecoverability from imputer-specific error, providing a useful conceptual basis for assessing when imputations can be trusted.
Critical weaknesses
-
The experimental evaluation is limited to the ETT series and one air-quality dataset, and the baselines are relatively outdated.
-
Figure 2, the main framework diagram, contains noticeable color inconsistencies across modules, leading me to suspect that generative AI may have been used to create it. However, the AI use statement does not explain in detail whether or how AI was used in preparing this figure.
-
The paper has substantial formatting and presentation issues, including large blank spaces around Section 5 and poor readability in parts of the appendix.
-
The paper emphasizes distinguishing insufficient contextual information from limitations of the imputer, yet the final selection is based on total risk. Regardless of the source of error, the action remains the same: retain or reject the imputation. The motivation for estimating these components separately is therefore insufficiently justified.
Questions for authors
See Weakness。
2 comments
Thank you for the response; however, my main concerns remain unresolved, so I will maintain my original rating.
Thank you for the critical review. We address the four weaknesses in turn, starting with the conceptual one, which we agree is central.
Why estimate the two components separately if the final action is retain or reject?
We give 2 answers both of which are supported by our experiments.
1. The components point to different remedies. For every test region we computed the error of the best of the seven imputers ("oracle switching") and the share of each imputer's error that switching would remove.
2. The decomposition also yields a better selection rule. Proposition 1 implies a training-free plug-in estimate of total risk: U^G plus the deviation DG,f of the imputer from CURE's conditional mean. It improves AURC over disagreement alone by 18.9% (95% CI 16.5–21.2), on every one of the six datasets, and over a learned direct predictor of total error with identical inputs by 18.1%. So even when the action is retain or reject, the decomposition matters because it tells us how to estimate the imputer-specific part.
Datasets and baselines.
Figure 2 and the AI-use statement. You are right. Figure 2 was generated with a generative AI tool, from a design and content specified by the authors, who checked that it depicts the method correctly. Our AI-use statement should have said so, and we apologize for the omission. In the revision we will declare this explicitly, fix the color inconsistencies, add a legend for every module, and extend the statement to cover any AI assistance with writing and code.
Formatting. In the revision we will remove the blank space around Section 5 and reorganize the appendix, with larger fonts in tables and figures and one appendix section per experiment.