- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the relationship between the effectiveness of missing value imputation and gains in prediction. Through controlled error allocation and selective replacement of imputed values, it finds that reconstruction improvements are not concentrated on predictively important features. The authors also explore whether restricting expensive imputation to important features saves computation without sacrificing predictive performance, and whether reconstruction error can guide imputer selection. Experiments cover regression and classification under simulated and natural missingness, measuring predictive performance and computational cost.
Critical strengths
1.The paper examines whether additional imputation cost brings better predictions on imputed data. Considering predictive performance, computational cost, and missingness mechanisms together provides a useful perspective beyond reconstruction accuracy alone. 2.The controlled experiments provide evidence about the relationship between feature importance and imputation gains. Evaluating this relationship across several models, tasks, and missingness settings offers useful empirical detail about when these gains occur.
Critical weaknesses
1.The motivation for accounting for feature importance in imputation is already studied, and the paper does not clearly explain what broader insight is brought in this paper. For example, Guo et al. [1] incorporate feature importance into the imputation objective, while xGEWFI [2] uses it to weight imputation errors during evaluation. These studies already motivate allocating greater attention to predictively important features. Besides, Section 8 finds that choosing an imputer by hiding some observed values and measuring reconstruction error can be unreliable. Näf et al. [3] already examine the practice of evaluating imputations by masking observed values and explain why it can produce incorrect rankings, even under MAR. The paper needs to explain what new understanding its experiments add beyond these established findings. 2.The writing does not clearly identify the main conclusions or build a coherent argument. Much of the paper lists numerical results without sufficiently explaining what readers should learn from them. The objectives, experimental settings, and conclusions of individual subsections are often unclear. The paper lacks an overarching analytical framework that distinguishes its central findings from supplementary observations. It needs substantial restructuring around a small number of conclusions, with a clear explanation of how each experiment supports them. 3. The experiments do not show whether targeted imputation remains useful on larger datasets or with missingness-aware tabular foundation models. The proposed procedure primarily uses a fixed fraction of features, and its benefits combine feature selection with fewer imputation iterations. Experiments cap datasets at 5,000 rows, leaving it unclear whether the relative benefits of full and targeted imputation persist at larger scales. More observations could improve imputation quality or help predictors use imputed values more effectively, potentially making the additional computation worthwhile. The current results therefore provide limited guidance on choosing between full and targeted imputation. The comparison also omits missingness-aware tabular foundation models such as TabPFN-3, which explicitly encode missingness and learn to predict from incomplete inputs through pretraining. Their omission limits the conclusions about neural predictors and the value of external imputation. [1] Cong Guo et al. Iterative missing value imputation based on feature importance. [2] Jean-Sébastien Dessureault et al. Explainable Global Error Weighted on Feature Importance: The xGEWFI metric to evaluate the error of data imputation and data augmentation. [3] Jeffrey Näf et al. How to rank imputation methods?
Questions for authors
- What is the main new finding relative to the related work discussed above? Which results establish this finding beyond further quantifying existing observations?
- How should practitioners choose the fraction of features to impute? The paper mainly uses the top third, but provides no general selection rule. How should this fraction be chosen for a new dataset, and how much does that choice affect the trade-off between predictive performance and computation?
- How does the predictive benefit of full versus targeted imputation change with sample size and dimensionality?
- Does external or targeted imputation still improve predictions for a missingness-aware model such as TabPFN-3? How would its behavior affect the paper’s interpretation of differences between model families?