- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates OT-based missing value imputation (MVI) problem. Specifically, it first introduces marginal optimal transport and derive loss functions based on modified Wasserstein distance. It also incorporates propensity weighting to mitigate sample selection bias. Experiments on synthetic and tabular benchmarks demonstrate the effectiveness of the proposed method.
Critical strengths
The problem of missing value imputation is clearly motivated and well formulated, and the introduction of optimal transport to address this problem is reasonable and feasible.
The proposed algorithms are clearly described in the main text and appendix. Authors also provide code in the supplementary file to improve reproducibility.
Critical weaknesses
-
OT-based imputation is a problem that attracts extensive investigation. Authors effectively discuss the original one (Muzellec et al., 2020) and one subsequent refinement (Jarrett et al., 2022), but ignore the existence of numerous literature in this field.
-
The experiment would benefit from comparing with more comprehensive and recent baselines. The baselines selected in this paper are somewhat out-of-dated, as all of them except KPI were presented before 2024. Moreover, only one OT-based imputation method is selected as baseline (Muzellec et al., 2020). These two factors make it insufficient to demonstrate the effectiveness of the proposed approach, given the rapid progression of this field.
-
The abstract noted that
samples can have no jointly observed coordinates and therefore represent different marginals, which is claimed as a limitation of existing OT methods. However, existing OT methods can address it by employing a preliminary imputer to generate initial value of imputations. Therefore, if authors want to demonstrate that this limitation persists, please consider the existence of preliminary imputer. Moreover, according to figure 1, the proposed method also resorts to a preliminary imputer, which makes it less persuasive that the proposed approach can separately address the identified limitation without preliminary imputation. -
It is known that preliminary imputer highly impacts the imputation performance. Authors should elaborate more on the implementation of preliminary imputer. Specifically, it is necessary to perform sensitivity analysis, by using different models as the preliminary imputer (e.g., mean value, mode value, etc) and compare the resulting performance with that of baselines.
-
The task to estimate propensity is truly difficult in practice. Moreover, if propensity is not accurately estimated, inverse propensity weighting induces bias. Authors should discuss the effectiveness of propensity score estimation accuracy in diverse scenarios, which is essential to back up the effectiveness of the method in practice.
-
The repository provided is not complete. At least a concise readme file and requirement.txt should be included, to initiate the project. The hyperparameters should also be attached to improve reproducibility.
-
Some theorems may be somewhat naive. For instance, Theorem 1 naturally holds, especially given the MCAR assumption.
-
The experiments would need further supports and analysis, including but not limited to experiments with different missing ratios, confidence intervals, efficiency and computational cost comparison, isolated and detailed ablation study, sensitivity analysis to OT hyperparameters, sensitivity to preliminary imputers, case study and demos of imputed tables, etc.
Questions for authors
Please see the comments above.