acceptodds
Under review as a conference paper at ICLR 2027

MissBGM: Missingness-aware Data Imputation via AI-powered Bayesian Generative Modeling

Abstract

Missing data imputation remains a fundamental challenge in modern data science, especially when uncertainty quantification is essential. In this work, we propose MissBGM, an AI-powered tabular data imputation method via Bayesian generative modeling that combines the flexibility of neural networks with the statistical rigor of Bayesian inference. Unlike existing methods that often focus on point estimates or treat the missingness mechanism implicitly, MissBGM jointly models the data-generating and missingness mechanisms explicitly, producing posterior distribution over each missing entry. We develop a novel stochastic optimization algorithm with alternating updates of missing values, model parameters, and latent variables until convergence. For the theoretical analysis, we establish convergence to a uniquely separated pseudo-true tempered target under explicit criterion, approximation, and optimization conditions. Empirically, MissBGM outperforms classical and neural imputers across a range of settings. These results establish MissBGM as a principled and scalable solution for modern missing data imputation.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Data imputation, deep generative model, Bayesian inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GN3kOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a Bayesian method (called MissBGM) for imputation of tabular missing data, specifically going beyond point-wise estimates and providing a detailed discussion on uncertainty quantification. A number of empirical results are provided (showing MissBGM in a very favorable light compared to classical methods), along with a theoretical analysis.

Critical strengths

Excellent empirical results. A number of baselines methods are included and the proposed method is clearly out-performant of them. There can be no doubt that the proposed method can outperform a wide range of as-is approaches that appear in the literature, on the tasks considered.

Consideration of an explicit model of missingness is also a main strength of this work. Such a consideration is not present in much of the literature, where the MCAR model is taken essentially as an off-the-shelf supervised learning problem. In this sense, the problem setting is a modern one, deserved of consideration.

Critical weaknesses

Critical Weakness: Unrefined framing of the problem, in particular wrt related work.

It is difficult to pin down a general suggestion on how to improve the framing, but the issue is visible in: "This challenge is particularly acute..." in the Introduction, where there is no challenge explicitly mentioned. There is no motivating application. The sentence "Unlike existing methods that often focus on point estimates" definitely implies the proposed method - since point estimates are often (not always) a focus. Language should be more scientific and exact.

It feels like there is a missing 2.3 Bayesian-based Imputation Methods for which there is definitely a wealth of literature to discuss. The positioning feels incomplete without this; particularly regarding the previous point implying that the main limitation of existing work is the focus on point estimates.

Without a motivating application, we should expect more than only a few datasets, or at least more modern (e.g., the Wine dataset is from 1991), it is is difficult to motivate modern AI-powered generative modelling for such data.

This is consequential to the assessment because it becomes difficult to judge the potential impact of the work in this arguably-saturated field on missing-data imputation.

Authors should reconsider sentences such as "we establish convergence to a uniquely separated pseudo-true tempered target under explicit criterion, approximation, and optimization conditions" in the abstract -- such claims require more specifics for the reader to understand, the reader should not be expected to read the entire paper in order to understand the a abstract.

Questions for authors

What exactly is identified under MNAR ?

Why does the theory discuss β=1\beta=1, while the method defaults to β\beta=0.01 ? (and appears very sensitive in this small range)

Why are fully Bayesian imputation methods absent from the Related Work and comparisons?

What are the computational costs in absolute terms for the datasets?

Is there evidence of downstream-task success from your proposed approach?

8 comments
AuthorsOpenReview ↗

Response to Reviewer GN3k (1/3)

We appreciate the reviewer's comments and suggestions. We address these points with additional experiments and have updated the paper and supplementary material. I apologize that I have to reply with three posts/replies here.

What exactly is identified under MNAR?

Without identifying restrictions, the observed-data distribution does not determine p(xmis∣xobs,r)p(x_{\mathrm{mis}}\mid x_{\mathrm{obs}},r) (Linero & Daniels, 2018). Theorem 1 establishes consistency for a criterion-induced imputation law, rather than general identification under MNAR. In particular, A6 assumes uniqueness and separation of that law. Recovery of the true law by the default profiled estimator additionally requires A8, including correct specification and Fisher consistency of the profiling criterion.

Possible MNAR identification conditions include (i) no self-censoring, Rj⊥Xj∣X−j,R−jR_j\perp X_j\mid X_{-j},R_{-j}, with the required positivity conditions (Sadinle & Reiter, 2017); (ii) Gaussian selection models under appropriate conditions (Miao, Ding & Geng, 2016); or (iii) valid shadow variables under appropriate conditions (Miao et al., 2024). We do not claim that our current model architecture automatically satisfies these conditions.

In our MNAR simulation design, the full-data and missing-value conditional distributions are identifiable. Given fully observed anchors AA, the target variables satisfy

R_j=\mathbf{1}\\{X_j\leq\mu_j(A)+\sigma_j\kappa\\},$$ where $\sigma_j > 0$ and the errors are independent standard Gaussian variables, independent of $A$. The observed response probability $P(R_j=1\mid A)=\Phi(\kappa)$ identifies $\kappa$. Writing $\lambda=\varphi(\kappa)/\Phi(\kappa)$, the observed conditional moments identify $$\sigma_j^2= \frac{\operatorname{Var}(X_j\mid A,R_j=1)} {1-\kappa\lambda-\lambda^2}, \qquad \mu_j(A)=E(X_j\mid A,R_j=1)+\sigma_j\lambda,$$ where the denominator is strictly positive. Thus, the observed-data law uniquely determines the Gaussian parameters and, under the stipulated conditional independence, the joint missing-value distribution through the corresponding upper-tail truncated Gaussian distributions. This establishes identification under our simulation assumptions. We have added this clarification to the revised paper. ## Role of $\beta$ in theory and practice Theorem 1 is formulated for a fixed $\beta$, including $\beta=0.01$, under A1–A6. It establishes convergence to the corresponding criterion-induced imputation law. The special role of $\beta=1$ is that it recovers the untempered model conditional. Identification and recovery of the true law require additional assumptions, including A8 for the profiled estimator. The default $\beta=0.01$ is an empirical choice for the original MAP procedure. To investigate the untempered alternative, we additionally implemented **MissBGM-S**, using $\beta=1$ in both training and inference. MissBGM-S replaces the MAP updates with alternating HMC updates of both $Z$ and $X_{\mathrm{mis}}$ conditional on the current parameters, making training consistent with the test stage. We then update $(\theta,\phi)$ using the average complete-data log-likelihood gradient over retained joint samples. MissBGM-S uses $\beta=1$ and a Bayesian neural network (use_bnn=True). We updated both the simulation and real-data benchmarks to include MissBGM-S in the revised paper. MissBGM-S reduced imputation RMSE and improved empirical coverage in all MNAR simulation settings, with improvements of 6.9%–25.7% relative to default MissBGM. On real datasets, MissBGM-S achieves slightly higher RMSE than the original MissBGM. We therefore retain both methods in the revised paper. We also updated the source code so the new results can be reproduced. These results demonstrate that improved MNAR performance is feasible without tempering ($\beta=1$).
AuthorsOpenReview ↗

Response to Reviewer GN3k (2/3)

Missing literature for Bayesian imputation methods

Thank you for highlighting established Bayesian imputation methods. Our original paper already cited Rubin (1987), Schafer (1997), and van Buuren and Groothuis-Oudshoorn (2011), and our baseline ICE used Bayesian-ridge predictive sampling.

To further strengthen the empirical comparison, we additionally implemented two Bayesian approachs: MICE-norm and sbgcop. MICE-norm propagates regression-parameter and predictive uncertainty through Bayesian normal-regression conditionals. sbgcop performs Bayesian Gaussian-copula inference for dependence parameters and missing latent values, with empirical marginal reconstruction on the observed scale. These provide two established Bayesian imputation approaches with different modeling assumptions.

We additionally benchmarked these two methods under both simulation and real data MNAR settings and compared them with MissBGM and MissBGM-S. Each table entry reports predictive-mean RMSE using 1,000 retained imputations per run. The run each method with 10 seeds and show the average RMSE.

MechanismnnMissing rateMissBGMMissBGM-SMICE-normsbgcop
MNAR50030%1.6201.2811.6231.621
MNAR50050%1.5851.2181.5941.588
MNAR50070%1.6731.2631.6771.677
MNAR1,00030%1.5681.2841.5761.580
MNAR1,00050%1.5381.1991.5481.544
MNAR1,00070%1.6191.2031.6291.625
MNAR5,00030%1.5991.2881.6011.590
MNAR5,00050%1.5741.3281.5781.576
MNAR5,00070%1.6621.5481.6721.671
DatasetMissBGMMissBGM-SMICE-normsbgcop
Wine0.6820.7250.7670.768
Breast0.5110.5760.5430.576
Concrete0.5290.5690.5990.693

MissBGM-S achieves lowest RMSE across all MNAR simulation settings and outperform the two baselines by a large margin. MissBGM achieves better performance than the best baseline in 8 out of 9 settings. In the MNAR real datasets, MissBGM is the still the best among all methods, outperforming the best baseline (MICE-norm) by 5.9–11.7%. These results again support the supriority of our MissBGM methods.

Computational cost

Thank you for requesting dataset-specific computational costs. Appendix H reports running times on a moderate size of dataset with n=5,000n=5{,}000, p=200p=200, and a missing rate of 20%20\%. All experiments were conducted on the same Linux-based high-performance computing cluster managed by Slurm. The compute nodes were equipped with NVIDIA RTX 5000 Ada GPUs and dual-socket CPUs with 48 CPU cores per node. For each experiment, we requested a maximum wall-clock time of 3 days and 300 GB of memory. We summarize the running time as below.

MethodRunning time (minutes)
Mean0.13
OT5.88
Miceforest215.60
ICE5.62
GAIN11.15
ForestDiffusion1,071.00
TabCSDI2,905.83
MissBGM16.17

The training and inference on a moderate size of dataset (n=5,000n=5{,}000, p=200p=200) of MissBGM only takes less than 17 minutes while a few deep imputers (e.g., ForestDiffusion, TabCSDI) will take more than 10 hours.

In supplementary experiments, we further compared MissBGM and MissBGM-S across the 12 MNAR simulation settings. Note that MissBGM-S uses short HMC chains during training (2 burn-in sweeps and 4 retained samples). The ratio of mean total runtime for MissBGM-S to MissBGM ranged from 1.11 to 1.35. On Wine, Breast, Concrete and Gisette, MissBGM-S required 1.4, 3.9, 6.1, and 201.2 minutes, respectively, compared with 1.1, 3.1, 4.8, and 164.2 minutes for MissBGM. Overall, the running time of MissBGM-S is 11-35% higher than MissBGM.

AuthorsOpenReview ↗

Response to Reviewer GN3k (3/3)

Downstream task

Thank you for highlighting the importance of downstream evaluation. We have conducted a series of downstream experiments to improve coefficient estimation in linear regression using different imputation methods.

We retain the data-generating process, with fixed population parameters and analytically known regression coefficients. We use Y=X1Y=X_1 as the fully observed outcome and X2,…,X50X_2,\ldots,X_{50} as predictors. Across 10 independent datasets of N=500N=500, we evaluate MNAR missing rates of 20% and 50%, with MCAR 20% as a reference. Missingness affects the 45 target predictors. The outcome is included in every imputation model. Each method provides 10 completed datasets, followed by separate OLS fits and Rubin pooling with Barnard–Rubin degrees of freedom for pointwise 95% confidence intervals. All configurations were fixed without task-specific tuning.

The table summarizes all 45 coefficients corresponding to potentially missing predictors. Bias and RMSE are calculated across datasets for each coefficient and then averaged across coefficients.

SettingMethodMean absolute biasMean coefficient RMSE ↓coverage rateMean interval width
MNAR 20%MissBGM0.004590.0169799.56%0.09746
MNAR 20%MissBGM-S0.004640.0173798.67%0.08935
MNAR 20%not-MIWAE0.005630.0232288.00%0.07551
MNAR 20%GINA0.011710.0253294.00%0.10381
MNAR 50%MissBGM0.010310.02276100.00%0.15416
MNAR 50%MissBGM-S0.011400.0245798.67%0.13040
MNAR 50%not-MIWAE0.010350.0314284.44%0.09184
MNAR 50%GINA0.029560.0562390.22%0.20168
MCAR 20%MissBGM0.003620.0158199.56%0.09267
MCAR 20%MissBGM-S0.004360.0162998.44%0.08465
MCAR 20%not-MIWAE0.005340.0212188.67%0.07259
MCAR 20%GINA0.008240.0216995.78%0.09479
Complete-data referenceOLS0.003450.0155894.89%0.06617

Default MissBGM achieves the lowest mean coefficient RMSE among all imputation methods in all three settings, reducing it over the best baseline (not-MIWAE), by 26.92%, 27.56%, and 25.46%, respectively. MissBGM-S also outperforms both baselines in mean coefficient RMSE and produces narrower intervals than default MissBGM.

These results provide direct evidence of downstream estimation of coefficient benefits from our MissBGM models. We will update the results in the revised paper.

Reviewer_gsNaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MissBGM, a Bayesian generative model for imputing missing tabular data that combines the expressivity of neural networks with Bayesian inference. Of particular note, MissBGM jointly models the data-generating and missingness mechanisms explicitly, seeking to produce posterior distribution over each missing entry under MNAR. Experimental results show that MissBGM outperforms classical and neural imputers across a range of settings, particularly under MNAR.

Critical strengths

Overall, the proposed modeling framework MissBGM (as described in Figure 1) is carefully designed; particularly the model for missingness indicator R includes both observed X and missing X and the algorithm involves alternating updating of parameters for the data model and the missingness model, enabling the learning of these two sets of parameters under MNAR. The extensive numerical experiments including those under MNAR demonstrate the advantages of MissBGM overall multiple baselines.

Critical weaknesses

Section 4.1: The proposed method assumes that continuous X follows multivariate normal distribution and p(x|z) has a diagonal covariate matrix. These assumptions likely would not hold in many real-world settings, so its usefulness in real word settings may be limited.

Section 4.2: The alternating stochastic optimization approach is not designed to simulate from the posterior distribution of (Z, X_{mis}, \theta, \phi). Particularly, in steps (i) and (ii), Z and X_{mis} are updated using the mode of their conditional distributions given the observed data and current values of the other parameters. Since it’s not really a full Bayesian approach, it unclear whether the proposed method still possesses appealing properties of a Bayesian approach, in say uncertainty quantification using the posterior predictive distributions of missing data for imputation; also see the next comment.

Section 4.3: Plugging in the estimated (\theta, \phi) from Section 4.2, the samples of X^{(s)}{mis} are not really the posterior distribution of X{mis} in the classical Bayesian sense. It’s unclear whether X^{(s)}_{mis} possesses properties that come directly from a classical Bayesian posterior inference.

Section 5: The theoretical results are for an idealized M-estimation. Its relevance to the implemented imputation method is unclear.

Section 6: There are no experimental results on the performance of the proposed imputation method for downstream tasks, in comparison with the current SoTA.

Questions for authors

I am unclear about the value of the experiments reported in Table 2, i.e., evaluating uncertainty quantification under MCAR masking. Would it be better to evaluate uncertainty quantification in downstream tasks under MNAR if the proposed method can be used for multiple imputation?

5 comments
AuthorsOpenReview ↗

Response to Reviewer gsNa (1/2)

We appreciate the reviewer's comments and suggestions. We address these points with extensive experiments as below. We also updated the paper and supplementary file.

Gaussian assumption for continuous X

This is a misread of our approach. We would like to clarify that the Gaussian assumption is conditional on the latent variable Z, rather than on the marginal distribution of X. Specifically, pθ(x)=∫N(x;μθ(z),diag⁡σθ2(z))π(z) dzp_\theta(x)=\int\mathcal N\left(x;\mu_\theta(z),\operatorname{diag}\sigma_\theta^2(z)\right)\pi(z)\ dz.

This is a continuous mixture of Gaussians, which can represent non-Gaussian, multimodal distributions with dependent features. Feature dependence is induced through the shared latent variable Z. The diagonal covariance assumption restricts only the residual dependence conditional on Z.

This conditional Gaussian decoder belongs to the same broad family used in latent variable models (LVMs), including not-MIWAE and GINA compared in our paper.

Regarding the fully Bayesian model

We agree that the default MissBGM is not fully Bayesian as it uses MAP for latent variable update during the training. To address this, we introduce MissBGM-S, a new variant of our default MissBGM by replacing the MAP update with alternates HMC updates of both ZZ and XmisX_{\mathrm{mis}} conditional on the current parameters, making the training consistent with the test stage. We then update (θ,ϕ)(\theta,\phi) using the average complete-data log-likelihood gradient over the retained joint samples. The update therefore retains conditional second-moment information instead of treating one completed batch as fully observed. The MissBGM-S always uses β=1\beta=1 and Bayesian neural net (use_bnn=True), and we also updated both the simulation and real-data benchmarks to include MissBGM-S in the revised paper. MissBGM-S reduced imputation RMSE and improved empirical coverage relative to MissBGM in all MNAR simulation settings. On the real datasets, MissBGM-S achieves higher RMSE than the original MissBGM. We therefore retain both MissBGM and MissBGM-S in the revised paper. We also update the source code to include MissBGM-S.

The newly implemented MissBGM-S is approximately Bayesian: the local variables (Z, X_mis) are sampled from their conditional posterior by HMC, and the network weights are given by a variational posterior. We do not claim exact full Bayesian inference since the weight posterior is a KL-weighted variational approximation.

Regarding the standard M-estimation

We agree that Theorem 1 concerns an idealized profiled M-estimator associated with default MAP training, rather than the finite-epoch algorithm. Its criterion maximizes over Z and XmisX_{\mathrm{mis}}. Under A1-A6, we establish convergence of induced missing-value conditional law to its population target in integrated total variation under the limiting observed-data marginal. The limiting law need not equal the true MNAR law as that interpretation requires additional Fisher-consistency and identification condition in A8. The theorem does not guarantee interval calibration or control finite-MCMC error.

Note that the new implemented variant MissBGM-S offers a more direct likelihood-based foundation. It approximates observed-data likelihood fitting by integrating out ZZ and XmisX_{\mathrm{mis}}, rather than maximizing over them and uses variational inference to account for parameters uncertainty. Under standard differentiation conditions, Fisher’s identity expresses the observed-data likelihood gradient as the conditional expectation of the complete-data score, providing a direct justification for the sampling-based updates. Under correct specification, the population likelihood is maximized at the true observed-data distribution through the KL-divergence identity. Note that this property does not by itself establish consistency of the variational estimator or identify the missing-value conditional distribution under unrestricted MNAR. Additional identifying restrictions, such as no self-censoring(Sadinle & Reiter, 2017) or valid shadow variables (Miao et al., 2024), remain necessary. We will clarify these distinctions in the revised paper.

AuthorsOpenReview ↗

Response to Reviewer gsNa (2/2)

Downstream tasks

To assess whether imputation performance translates into downstream benefits, we conducted linear regression coefficient experiments under the original 50-variable data-generating model, with fixed population parameters and analytically known linear regression coefficients.

We use Y=X1Y=X_1 as the fully observed outcome and X2,…,X50X_2,\ldots,X_{50} as predictors. Across 10 independent datasets of N=500N=500, we evaluate MNAR missing rates of 20% and 50%, with MCAR 20% as a reference. Missingness affects the 45 target predictors. The outcome is included in every imputation model. Each method provides 10 completed datasets, followed by separate OLS fits and Rubin pooling with Barnard–Rubin degrees of freedom for pointwise 95% confidence intervals. All configurations were fixed without task-specific tuning.

The table summarizes all 45 coefficients corresponding to potentially missing predictors. Bias and RMSE are calculated across datasets for each coefficient and then averaged across coefficients.

SettingMethodMean absolute biasMean coefficient RMSE ↓coverage rateMean interval width
MNAR 20%MissBGM0.004590.0169799.56%0.09746
MNAR 20%MissBGM-S0.004640.0173798.67%0.08935
MNAR 20%not-MIWAE0.005630.0232288.00%0.07551
MNAR 20%GINA0.011710.0253294.00%0.10381
MNAR 50%MissBGM0.010310.02276100.00%0.15416
MNAR 50%MissBGM-S0.011400.0245798.67%0.13040
MNAR 50%not-MIWAE0.010350.0314284.44%0.09184
MNAR 50%GINA0.029560.0562390.22%0.20168
MCAR 20%MissBGM0.003620.0158199.56%0.09267
MCAR 20%MissBGM-S0.004360.0162998.44%0.08465
MCAR 20%not-MIWAE0.005340.0212188.67%0.07259
MCAR 20%GINA0.008240.0216995.78%0.09479
Complete-data referenceOLS0.003450.0155894.89%0.06617

Default MissBGM achieves the lowest mean coefficient RMSE among all imputation methods in all three settings, reducing it over the best baseline (not-MIWAE), by 26.92%, 27.56%, and 25.46%, respectively. MissBGM-S also outperforms both baselines in mean coefficient RMSE and produces narrower intervals than default MissBGM.

These results support a downstream coefficient-estimation benefit under the evaluated configurations. They also show that lower imputation RMSE does not necessarily translate into lower regression-coefficient RMSE.

Uncertainty evaluation under MNAR

Thank you for this suggestion. We have conducted direct uncertainty comparisons with not-MIWAE and GINA, using 1,000 predictive completions to construct 95% intervals. Under MCAR, we already showed MissBGM achieves coverage closest to nominal.

To benchmark the MNAR on uncertainty quantification, we extended the MNAR mechanism described in Appendix C.1 to all 12 combinations of n∈{500,1000,5000,10000}n \in \{500, 1000, 5000, 10000\} and r∈{0.3,0.5,0.7}r \in \{0.3, 0.5, 0.7\}. We use p=50p = 50.

Each entry below reports coverage (%) / mean interval width for nominal 95% intervals. Results are averaged over 10 model seeds on fixed data and masks within each setting, using 1,000 predictive completions per run.

nnrrMissBGMMissBGM-Snot-MIWAEGINA
5000.336.98 / 2.51756.20 / 2.3554.75 / 0.54819.48 / 1.444
5000.532.01 / 2.10959.54 / 2.1334.64 / 0.51924.30 / 1.561
5000.728.03 / 1.72363.84 / 2.3774.95 / 0.45326.05 / 1.621
10000.335.97 / 2.44352.03 / 2.0879.75 / 1.13122.82 / 1.685
10000.529.13 / 2.03059.42 / 2.2748.69 / 1.02425.74 / 1.609
10000.721.65 / 1.70763.19 / 2.5158.06 / 0.85726.16 / 1.672
50000.330.61 / 2.38062.88 / 2.63834.81 / 2.43410.52 / 0.953
50000.524.29 / 1.96858.16 / 2.60125.52 / 2.10913.51 / 1.035
50000.720.96 / 1.67048.45 / 2.57519.44 / 1.80319.95 / 1.348
100000.329.70 / 2.30159.82 / 2.67836.75 / 2.4017.69 / 0.759
100000.522.85 / 1.88259.08 / 2.55227.03 / 2.09111.91 / 0.908
100000.717.58 / 1.59651.16 / 2.65520.51 / 1.78916.95 / 1.178

All evaluated methods substantially undercover under this challenging MNAR mechanism. Default MissBGM outperforms GINA in 11 of the 12 settings. The new variant, MissBGM-S, achieves the highest coverage in every setting, exceeding the higher-coverage baseline in each setting by 23.07–37.79 percentage points. It also achieves lower interval scores than both baselines in every setting, accounting jointly for interval width and noncoverage.

These results demonstrate the miscalibration of all exsiting methods, but MissBGM methods achieve relatively the best. We updated the results in revised paper.

I hope that my additional experiments and clarifications have addressed your major concerns. I would greatly appreciate your consideration of updating the assessment and score accordingly. I would also be happy to discuss and address any further questions that you may have. Thank you.

Reviewer_wTRgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MissBGM for tabular data imputation. It combines a latent-variable data model with a neural model of the missingness process. Training alternates between optimizing latent variables, missing entries, and network parameters. After training, HMC within Gibbs samples latent variables and missing entries with the networks fixed. The paper also gives a consistency result for an idealized profiled estimator. Experiments report improved imputation accuracy on synthetic data and four real datasets, together with an uncertainty evaluation under MCAR.

Critical strengths

S1: The method explicitly models both the data and the missingness process. This allows the mask to inform imputation when missingness may depend on unobserved values.

S2: The HMC stage explores the fitted conditional distribution without an amortized encoder. Its samples support multiple imputations and uncertainty summaries beyond a single point estimate.

S3: The point-imputation results are promising, with the lowest mean RMSE on all four real datasets. The MCAR experiment also reports coverage close to the nominal level.

Critical weaknesses

W1: Point optimization can bias the fitted uncertainty. In an unpenalized Gaussian submodel with known zero mean and a constant mask model, fully optimizing the completed-data likelihood gives σ^2→(1−r)σ02\hat\sigma^2\to(1-r)\sigma_0^2, where rr is the MCAR missing rate. Appendix B acknowledges possible profile bias, but does not show how the proposed training controls it.

W2: The effective prior under tempering needs clarification. Eq. (8) weights the mask likelihood by β\beta, while Eq. (9) leaves it untempered relative to the prior. With the L2 penalty in Appendix E.1, a common objective matching these updates would also scale the mask penalty by β\beta. This implied change in prior weight should be made explicit.

W3: The uncertainty advantage over variational MNAR models is not directly tested. Section 2.2 discusses their potential underestimation of uncertainty, yet Table 2 omits not-MIWAE and GINA. The MNAR undercoverage acknowledged in Appendix C.1 is also not quantified.

Questions for authors

Q1: Given the completed-batch updates described in Appendix E.3, how does training control profiling-induced variance shrinkage? Can this be checked against observed-data likelihood fitting in the Gaussian submodel?

Q2: Do the updates correspond to a common objective with a β\beta-scaled mask penalty? Please state the effective prior and its relation to Eqs. (8)–(9) for the regularized implementation.

Q3: How does MissBGM compare with not-MIWAE and GINA in coverage and interval width? Please also quantify the MNAR stress-test results mentioned in Appendix C.1.

3 comments
AuthorsOpenReview ↗

Response to Reviewer wTRg (1/2)

We appreciate the reviewer's comments and suggestions. We address these points with extensive experiments as below (two posts). We also updated the paper and supplementary file.

W1/Q1:Profiling-induced variance shrinkage

We agree that jointly maximizing the completed-data likelihood over (Xmis,θ)(X_{\mathrm{mis}},\theta) is the classical inconsistent joint-maximization alternative to observed-data maximum likelihood (Little & Rubin, 2019). The original MAP training does not provide a general correction for this. Subsequent Hamiltonian Monte Carlo (HMC) with fixed fitted parameters explores the fitted distribution, but cannot repair a variance underestimated during training.

To address this mechanism directly, we introduce MissBGM-S, a new variant of MissBGM by replacing the MAP update with alternates HMC updates of both ZZ and XmisX_{\mathrm{mis}} conditional on the current parameters, making the training consistent with the test stage. We then update (θ,ϕ)(\theta,\phi) using the average complete-data log-likelihood gradient over the retained joint samples. The update therefore retains conditional second-moment information instead of treating one completed batch as fully observed. The MissBGM-S always uses β=1\beta=1 and Bayesian neural net (use_bnn=True) and we also updated both the simulation and real-data benchmarks to include MissBGM-S in the revised paper.

For the reviewer's unpenalized Gaussian submodel with known mean zero and a constant MCAR mask with n=5000n=5000, and true variance σ02=1\sigma_0^2=1. Entries are averages over 100 independent datasets at each missing rate. Observed-data likelihood fitting (Observed ML) suggested by the reviewer is implemented. As shown in the table below. MissBGM’s hard-profile underestimates the variance. In contrast, our MissBGM-S closely matches Observed ML, yielding variance estimates near one and coverage near the nominal 95% across all three missing rates. These results confirm the reviewer’s diagnosis and demonstrate that MissBGM-S effectively corrects this specific source of variance shrinkage in the Gaussian submodel.

rrVariance: Observed MLVariance: MissBGMVariance: MissBGM-S95% coverage: Observed ML95% coverage: MissBGM95% coverage: MissBGM-S
0.20.9960.7960.99694.8491.8194.84
0.50.9970.4980.99794.9883.4194.98
0.70.9940.2990.99494.9171.7194.91

Beyond this, MissBGM-S also reduced imputation RMSE and improved empirical coverage relative to MissBGM in all MNAR simulation settings. On the real datasets, MissBGM-S achieves higher RMSE than the original MissBGM. We therefore retain both MissBGM and MissBGM-S in the revised paper and update the source code to include MissBGM-S.

W2/Q2:Clarification on objective functions

The default MissBGM does not include global-parameter L2 penalties. Only when the optional BNN is enabled, the Gaussian negative-log-prior terms of the biases give L2 penalties. We made it more clear in revised Appendix E1.

For the default, unpenalized implementation and β>0\beta > 0,

arg⁡max⁡ϕlog⁡pϕ(R∣X)=arg⁡max⁡ϕβlog⁡pϕ(R∣X).\arg\max_{\phi} \log p_{\phi}(R \mid X) = \arg\max_{\phi} \beta \log p_{\phi}(R \mid X).

This equality concerns exact block maximizers. Thus, omitting the positive factor β\beta from the implemented ϕ\phi-subproblem does not change its exact maximizers. We clarified this distinction alongside Eq. (9).

AuthorsOpenReview ↗

Response to Reviewer wTRg (2/2)

W3/Q3 Uncertainty experiments in MNAR

Thank you for this suggestion. We have conducted direct uncertainty comparisons with not-MIWAE and GINA, using 1,000 predictive completions to construct 95% intervals. Under MCAR, we added not-MIWAE and GINA, and the updated table 2 in the paper is shown below. MissBGM achieves the lowerest RMSE while MissBGM-S achieves the highest coverage among comparison methods.

MethodRMSE ↓WRCoverage
Miceforest1.2250.7740.287
GAIN1.0870.0030.002
ForestDiffusion0.9600.8590.885
TabCSDI1.2620.0710.068
not-MIWAE0.9570.9910.928
GINA1.1220.2750.324
MissBGM0.9460.9940.934
MissBGM-S0.9520.9960.936

To benchmark the MNAR on uncertainty quantification, we extended the MNAR mechanism described in Appendix C.1 to all 12 combinations of n∈{500,1000,5000,10000}n \in \{500, 1000, 5000, 10000\} and r∈{0.3,0.5,0.7}r \in \{0.3, 0.5, 0.7\}. We use p=50p = 50.

Each entry below reports coverage (%) / mean interval width for nominal 95% intervals. Results are averaged over 10 model seeds on fixed data and masks within each setting, using 1,000 predictive completions per run.

nnrrMissBGMMissBGM-Snot-MIWAEGINA
5000.336.98 / 2.51756.20 / 2.3554.75 / 0.54819.48 / 1.444
5000.532.01 / 2.10959.54 / 2.1334.64 / 0.51924.30 / 1.561
5000.728.03 / 1.72363.84 / 2.3774.95 / 0.45326.05 / 1.621
10000.335.97 / 2.44352.03 / 2.0879.75 / 1.13122.82 / 1.685
10000.529.13 / 2.03059.42 / 2.2748.69 / 1.02425.74 / 1.609
10000.721.65 / 1.70763.19 / 2.5158.06 / 0.85726.16 / 1.672
50000.330.61 / 2.38062.88 / 2.63834.81 / 2.43410.52 / 0.953
50000.524.29 / 1.96858.16 / 2.60125.52 / 2.10913.51 / 1.035
50000.720.96 / 1.67048.45 / 2.57519.44 / 1.80319.95 / 1.348
100000.329.70 / 2.30159.82 / 2.67836.75 / 2.4017.69 / 0.759
100000.522.85 / 1.88259.08 / 2.55227.03 / 2.09111.91 / 0.908
100000.717.58 / 1.59651.16 / 2.65520.51 / 1.78916.95 / 1.178

All evaluated methods substantially undercover under this challenging MNAR mechanism. Default MissBGM outperforms GINA in 11 of the 12 settings. The new variant, MissBGM-S, achieves the highest coverage in every setting, exceeding the higher-coverage baseline in each setting by 23.07–37.79 percentage points. It also achieves lower interval scores than both baselines in every setting, accounting jointly for interval width and noncoverage.

These results demonstrate the several miscalibration of all evaluated methods, but our MissBGM methods achieve relatively the best coverage among the comparison baselines. We updated the results in the revised paper.

I hope that my additional experiments and clarifications have addressed your major concerns. I would greatly appreciate your consideration of updating the assessment and score accordingly. I would also be happy to discuss and address any further questions that you may have. Thank you.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.