acceptodds
Under review as a conference paper at ICLR 2027

Imputers Improve the Wrong Features: Placement, Not Reconstruction Error Alone, Governs What Better Imputation Buys

Abstract

Imputers are compared by reconstruction error, and better reconstruction is assumed to buy better prediction. In practice the gain is small, and this paper measures where the improvement goes. Holding the total reconstruction error fixed and moving it across features changes downstream accuracy substantially, because predictive importance is concentrated in a minority of features, while the standard imputers we evaluate spread their improvement evenly and place only about a third of that improvement where the importance sits. We then attribute the gain using the imputations themselves: replacing only the important third's imputed values recovers most of what a better imputer is worth while spending a third of the error reduction, whereas the same total reduction placed on unimportant features recovers only a fraction. This holds at two missingness rates, on classification, and, with the sign reversed, under informative missingness, where better imputation instead costs tree ensembles. The diagnosis yields a procedure that imputes only where importance sits, at a fraction of the cost when the imputer is expensive and with no loss we can detect. Finally, treating imputer choice as a decision, we find that the diagnostic practitioners can compute is unreliable precisely when the missingness is informative, and is then beaten by a default fixed in advance. Better imputation is not failing to improve reconstruction; the improvement lands where the model cannot use it.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
missing data, imputation, imputer selection

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hbGAOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the relationship between the effectiveness of missing value imputation and gains in prediction. Through controlled error allocation and selective replacement of imputed values, it finds that reconstruction improvements are not concentrated on predictively important features. The authors also explore whether restricting expensive imputation to important features saves computation without sacrificing predictive performance, and whether reconstruction error can guide imputer selection. Experiments cover regression and classification under simulated and natural missingness, measuring predictive performance and computational cost.

Critical strengths

1.The paper examines whether additional imputation cost brings better predictions on imputed data. Considering predictive performance, computational cost, and missingness mechanisms together provides a useful perspective beyond reconstruction accuracy alone. 2.The controlled experiments provide evidence about the relationship between feature importance and imputation gains. Evaluating this relationship across several models, tasks, and missingness settings offers useful empirical detail about when these gains occur.

Critical weaknesses

1.The motivation for accounting for feature importance in imputation is already studied, and the paper does not clearly explain what broader insight is brought in this paper. For example, Guo et al. [1] incorporate feature importance into the imputation objective, while xGEWFI [2] uses it to weight imputation errors during evaluation. These studies already motivate allocating greater attention to predictively important features. Besides, Section 8 finds that choosing an imputer by hiding some observed values and measuring reconstruction error can be unreliable. Näf et al. [3] already examine the practice of evaluating imputations by masking observed values and explain why it can produce incorrect rankings, even under MAR. The paper needs to explain what new understanding its experiments add beyond these established findings. 2.The writing does not clearly identify the main conclusions or build a coherent argument. Much of the paper lists numerical results without sufficiently explaining what readers should learn from them. The objectives, experimental settings, and conclusions of individual subsections are often unclear. The paper lacks an overarching analytical framework that distinguishes its central findings from supplementary observations. It needs substantial restructuring around a small number of conclusions, with a clear explanation of how each experiment supports them. 3. The experiments do not show whether targeted imputation remains useful on larger datasets or with missingness-aware tabular foundation models. The proposed procedure primarily uses a fixed fraction of features, and its benefits combine feature selection with fewer imputation iterations. Experiments cap datasets at 5,000 rows, leaving it unclear whether the relative benefits of full and targeted imputation persist at larger scales. More observations could improve imputation quality or help predictors use imputed values more effectively, potentially making the additional computation worthwhile. The current results therefore provide limited guidance on choosing between full and targeted imputation. The comparison also omits missingness-aware tabular foundation models such as TabPFN-3, which explicitly encode missingness and learn to predict from incomplete inputs through pretraining. Their omission limits the conclusions about neural predictors and the value of external imputation. [1] Cong Guo et al. Iterative missing value imputation based on feature importance. [2] Jean-Sébastien Dessureault et al. Explainable Global Error Weighted on Feature Importance: The xGEWFI metric to evaluate the error of data imputation and data augmentation. [3] Jeffrey Näf et al. How to rank imputation methods?

Questions for authors
  1. What is the main new finding relative to the related work discussed above? Which results establish this finding beyond further quantifying existing observations?
  2. How should practitioners choose the fraction of features to impute? The paper mainly uses the top third, but provides no general selection rule. How should this fraction be chosen for a new dataset, and how much does that choice affect the trade-off between predictive performance and computation?
  3. How does the predictive benefit of full versus targeted imputation change with sample size and dimensionality?
  4. Does external or targeted imputation still improve predictions for a missingness-aware model such as TabPFN-3? How would its behavior affect the paper’s interpretation of differences between model families?
Reviewer_KoCLOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies why better reconstruction often yields little improvement in prediction. It shows that reconstruction improvements are spread across features, while predictive importance is concentrated, and uses partial replacement of real imputations to measure where predictive gains originate. It then proposes imputing only important columns and evaluates imputer-selection rules under different missingness mechanisms. Experiments cover 19 regression and 15 classification datasets with simulated missingness, plus 23 naturally incomplete datasets.

Critical strengths
  1. The partial-replacement experiment connects the explanation to actual imputers.

  2. The runtime analysis includes the cost of estimating importance and identifies when targeting saves nothing.

Critical weaknesses
  1. The advance over existing work remains limited. The paper correctly credits Le Morvan and Varoquaux (2025) for diminishing returns and mask effects, and Guo et al. (2024) already incorporate feature importance into imputation. The attribution experiments add useful evidence, but a prediction-quality and runtime comparison with the latter approach is needed to establish the proposed procedure's distinct value.

  2. This paper does not show how much larger that gain could become by reallocating an attainable reconstruction improvement. Section 3 acknowledges this limit, but the conclusion still calls placement a major reason for small gains without quantifying how much of the reconstruction–prediction gap it explains.

  3. The schedule ablation is helpful, but a full-column, single-pass baseline is still needed to isolate targeting from simply running a cheaper imputer. An equal-budget random-column comparison with the expensive random-forest imputer would also help establish whether importance ranking preserves more predictive value at the same cost.

  4. The practical benefit is least secure for the models that require imputation. Native missing-value handling outperforms targeting for trees, while targeting reduces the MLP's performance with the random-forest imputer, and no targeted FT-Transformer or SAINT result is reported.

Questions for authors
  1. Section 8 and Table 10 report different validation-selection results. The text gives the linear model a gain of +0.008 AUC on eight classification datasets, whereas Table 10 uses eleven and reports +0.005. The text also describes losses for every tree ensemble, while the table gives XGBoost approximately zero gain.

  2. The dataset-screening counts in Appendix A do not add up. The text says five near-duplicates were excluded, but lists two Fraud-Detection copies, three albert subsamples, and two communities-and-crime variants, totaling seven.

Reviewer_gNgjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks why better imputation yields small downstream gains even under MCAR. Its central claim is that placement, not reconstruction error alone, governs what better imputation buys. Holding total reconstruction error fixed and moving it across features changes downstream accuracy substantially. Predictive importance is concentrated in a minority of features, while standard imputers spread improvement roughly evenly and place only about a third of it where importance sits. The paper supports this with two designs. First, it injects a fixed error budget and varies only its allocation. Second, it replaces the imputed values of a chosen feature subset with a better imputer's values, using a greedily matched unimportant control. It then proposes targeted imputation, which ranks columns by permutation importance on a mean-imputed model and runs the expensive imputer only on the top third. The procedure matches full imputation in most (model, imputer) pairs, beats it in two, loses in one, and saves time when the imputer is expensive. Finally, the paper evaluates imputer selection as a decision. The held-out-entry diagnostic is accurate under MCAR and fails under MNAR, where a fixed leave-one-dataset-out default beats it.

Critical strengths

This paper reports that importance weighted selection is not helpful, as under MNAR, retained entries fail to diagnose and fixed default values defeat diagnoses outside of MCAR. These are useful findings that are often overlooked in literature. This paper explicitly separates its replication of LeMorvan&Varoquaux (2025) and Perez Lebel et al. (2022) from its own contributions. This article covers simulation and natural deficiency, regression and classification, seven model families, and multiple inputs. It also includes robustness checks for k/d and natural missing settings.

Critical weaknesses
  1. The headline attribution resolves for only two of four model families at the main setting. At MCAR 50%, the important-third versus matched-control contrast is significant for ridge and MLP, and unresolved for XGBoost and HistGB because their total gains are too small (+0.012 and +0.003).
  2. The practical payoff is mostly cost, not accuracy. In the regression/MCAR setting, targeted imputation is statistically indistinguishable from full imputation in five of eight pairs, better in two, and worse in one. The paper calls this "no detected loss" rather than equivalence, which is correct but limits the contribution. For the cheap iterative imputer, targeting is a net loss except for ridge. The method's value therefore depends on the expensive-imputer regime.
  3. Targeted imputation confounds two changes. It uses fewer columns and a single pass instead of the full iterative schedule. The third arm attempts to separate these, but neither component is significant at n=19. The result should be described as a procedure-level effect, not attributed to the importance ranking alone.
  4. No deep imputers are evaluated. The candidate pool is limited to mean, KNN, iterative, random-forest, and low-rank methods.
Questions for authors

see weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.