acceptodds
Withdrawn from ICLR 2027

Missing Data Imputation under Manifold Hypothesis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The manifold hypothesis posits that high-dimensional data are concentrated near a low-dimensional embedded manifold. Recent advances in mixture variational autoencoders (VAEs) provide a powerful tool for extracting such underlying structure in a faithful manner. The resulting geometric structure naturally introduces local and global relationships among variables, thereby providing a systematic way of imputing missing data. We propose a model-based imputation method that enables sampling from via a sampling-importance-resampling (SIR) procedure, which can be further augmented with a joint diffusion model in the latent space. Our method imputes missing data while respecting the underlying geometry, achieves competitive performance compared to state-of-the-art procedures, quantifies uncertainty in the imputations, and is model-based, thereby enabling on-the-fly imputation without rerunning the entire procedure.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
missing data imputation, manifold learning, Bayesian probability, latent diffusion

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_gvTZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the problem of missing data imputation for continuous tabular data, motivated by the manifold hypothesis that high-dimensional data often concentrate near low-dimensional manifolds. The authors propose a model-based imputation framework that employs a mixture of variational autoencoders (VAEs) to learn multiple latent spaces and uses sampling-importance-resampling (SIR) to select latent samples based on the likelihood of observed values for missing data reconstruction. Additionally, the framework incorporates a diffusion model that jointly models continuous latent variables and discrete chart labels to enhance latent sample generation.

Critical strengths
  1. The paper presents an intuitive and insightful approach by connecting the manifold hypothesis with conditional imputation from a geometric modeling perspective.
  2. The proposed method can generate multiple candidate imputations for the same observed sample, while explicitly discussing its reliance on complete reference data and its applicability to continuous variables.
Critical weaknesses

1.The experiments lack comparisons with representative recent imputation methods, such as HyperImpute (ICML 2022), HH-VAEM (NeurIPS 2022), ReMasker (ICLR 2024), ForestDiffusion (AISTATS 2024), and DiffPuter (ICLR 2025), making it difficult to adequately assess the competitiveness of the proposed method. 2.The paper lacks a dedicated ablation study of the latent diffusion module, leaving its effectiveness and specific contribution to the overall performance insufficiently validated. 3.The theoretical guarantee in Theorem 1 relies on idealized assumptions that the model is known and samples can be drawn from the true latent prior distribution, which differ from the practical setting where learned approximate models are used. Therefore, the applicability of this guarantee to the actual implementation remains unclear.

Questions for authors

1.The authors should clarify the generation procedures for MCAR, MAR, and MNAR missingness masks. I also suggest reporting standard deviations or confidence intervals for the main experimental results to improve their reliability. 2.I encourage the authors to include comparisons with representative recent imputation methods, as mentioned in W1, to provide a more comprehensive evaluation of the proposed method's competitiveness. 3.How does Mixture-VAE + SIR perform when the candidate pool consists only of training-set encoded latent samples without latent diffusion? Additionally, how sensitive are the imputation performance and effective sample size (ESS) to the candidate pool size and decoder variance? 4.For the temporally ordered Energy and Finance datasets, could the authors clarify whether chronological data splits are used? Additionally, please provide a detailed description of the complete experimental pipeline for the downstream regression task. 5.Does the proposed method remain effective when the training data contain missing values or when the proportion of complete samples is reduced? I suggest conducting additional experiments under these settings to evaluate the method's robustness and practical applicability.

Reviewer_S9bNOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper combined mixture variational autoencoders and a diffusion-based imputation method to obtain better performance on data imputation.

Critical strengths
  • The paper is easy to follow.
Critical weaknesses
  • The experiment is extremely limited; I strongly recommend validation on a large-scale dataset, even if it is not purely manifold-like distributed.
  • Experimental comparison with other diffusion-based imputation methods, such as [1].
  • The manifold hypothesis only motivates the adoption of mixture variational autoencoders, but does not motivate other novel designs.

[1] Zhichao Chen, Haoxuan Li, Fangyikang Wang, Odin Zhang, Hu Xu, Xiaoyu Jiang, Zhihuan Song, and Hao Wang. Rethinking the diffusion models for missing data imputation: A gradient flow perspective. In Advances in Neural Information Processing Systems, pp. 112050–112103, 2024.

Questions for authors

see weakness

Reviewer_5VD2OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a missing data imputation framework that combines existing methods, including Mixture VAE based manifold learning, Joint Latent Diffusion, and Sampling-Importance-Resampling (SIR). During training, a Mixture VAE, which uses multiple encoder-decoder pairs, is trained to represent the data using continuous latent variables and discrete chart labels, where each chart label indicates which pair is used to represent a data point in the latent space. Noise is then added to the latent variables and chart labels, and Joint Latent Diffusion is trained to predict their original values. During inference, the trained Joint Latent Diffusion generates new latent variables and chart labels from random noise, and SIR probabilistically selects candidates that best match the observed values. The selected candidates are then passed through the Mixture VAE decoder to impute the missing values. The method is evaluated on three synthetic and six real-world datasets using RMSE, Wasserstein distance, and downstream task performance. On synthetic datasets, it achieves the lowest Wasserstein distance but the highest RMSE, while on real-world datasets, it outperforms existing methods under some datasets and missingness conditions but underperforms them under the other cases.

Critical strengths
  1. The paper uses a simple example and visualization in Figure 1 to illustrate the difference between imputation results that preserve the underlying manifold structure and those that do not. This helps readers understand the motivation behind the study.

  2. The method is evaluated on three synthetic and six real-world datasets under MCAR, MAR, and MNAR missingness mechanisms, using RMSE, Wasserstein distance, and downstream task performance. This broad evaluation provides a useful basis for assessing not only pointwise imputation accuracy but also distributional fidelity and practical utility.

Critical weaknesses
  1. The research gap and methodological novelty are not sufficiently established.
  • The Introduction says that existing missing data imputation methods have not yet connected the manifold hypothesis with missing data imputation. However, prior studies such as "Manifold Learning and Missing Data Recovery through Unsupervised Regression" (ICDM 2011) and "A Denoising View of Matrix Completion" (NeurIPS 2011) have already explored missing data recovery using manifold structures.
  • The proposed framework primarily combines existing techniques, including Mixture VAE based manifold learning, SIR, and diffusion modeling of continuous and discrete variables. It remains unclear what fundamentally new algorithmic contributions are introduced beyond combining these techniques.
  1. The experimental evidence is insufficient to establish the effectiveness and necessity of the proposed framework.
  • The paper lacks sufficient quantitative ablation studies to establish the contributions of Joint Latent Diffusion and the multi-chart structure to imputation performance.
  • While the framework relies on the manifold hypothesis, the paper does not adequately evaluate how well the learned manifold representations capture real-world data or how their quality relates to imputation performance.
  • The experimental results exhibit substantially different performance across datasets, missingness mechanisms, and evaluation metrics. Although the authors provide some explanations, they do not sufficiently explain or systematically analyze these performance differences, making it difficult to determine under which conditions the method is effective.
  • The baseline comparisons are limited to methods published up to 2018, with no comparisons against recent probabilistic or diffusion-based imputation methods.
  • The paper lacks quantitative evaluations of computational costs.

** There is a grammatical error, "Information on the datasets are provided".

Questions for authors
  1. What is the methodological novelty of the proposed framework beyond combining existing techniques?

  2. How well do the learned manifold representations capture the structure of real-world data, and how does their quality relate to imputation performance? Can the authors provide quantitative evaluations of the learned representations and investigate whether the geometric regularization contributes to improvements in imputation performance?

  3. Can the authors provide quantitative ablation studies to demonstrate the contributions of Joint Latent Diffusion and the multi-chart structure to imputation performance? In particular, how does the performance differ between (1) SIR using latent samples directly from the Mixture VAE and SIR using samples generated by Joint Latent Diffusion, and (2) single-chart and multi-chart representations?

  4. What explains the substantial performance differences across datasets, missingness mechanisms, and evaluation metrics? Can the authors provide clearer explanations and a more systematic analysis of these differences to identify the conditions under which the proposed method is effective?

  5. How does the proposed method compare with recent probabilistic and diffusion-based imputation methods in terms of imputation performance? Can the authors also provide quantitative comparisons of computational efficiency with existing baselines?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.