- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
In some datasets, a value is often missing not because it is randomly missing, but because it was on purpose not recorded, maybe because the data was not considered useful. This means that a latent variable is responsible for its missing. The paper calls this indication-driven missingness. The authors consider that standard MNAR missing-not-at-random) imputers only models whether a value was observed, and thus cannot separate “measured and observed” from “intended to be measured but missing” from “never intended to be measured.” The proposed method, PU-VAE-MNAR, treats the latent measurement-intent indicator as a positive-unlabeled (PU) learning problem: observed entries are known positives, everything else is unlabeled. The authors use a per-variable adversarial PU classifier to turn fully observed covariates into a soft intent probability. This latter which conditions a VAE-based MNAR imputer. Imputed data are fed to a downstream logistic-regression predictor.
Critical strengths
The paper is novel in its framing and the treating of latent measurement intent as a PU-learning problem and using it to condition an MNAR VAE. The components (PAN, not-MIWAE-style mask decoding, conditional priors) are existing building blocks. The main novelty is in their combination and in the indication-driven formulation. The paper is well written and uses a large, realistic data (106k patients, 167 labs, 92% missingness), which is far bigger and more realistic than the small UCI tables common in deep imputation papers. The mix of continuous and binary labs is also handled explicitly. The paper evaluates together imputation and prediction., while many imputation papers report only RMSE. The authors of this also measures downstream prediction. The paper compares against 10 imputers across four families (statistical, MissForest, GAN, VAE/MNAR), including recent MNAR-specific models (not-MIWAE, GINA, PSMVAE, GNR), with links to the code for reproductibility. The paper is honest about its weaknesses (below mean imputation at extreme sparsity) and includes a detailed appendix (hyperparameter ranges, architectures, simulation equations). The authors do also provide an explicit AI-use statement.
Critical weaknesses
I could not really grasp from the paper how important the problem really is in practice. While missing data is a real problem and imputation techniques are really needed in practice. It's not clear to which extent using an imputation technique on healthcare related data is legally acceptable to use in medical sciences.
The paper uses a non public real world dataset, which is frustrating. It should have used a publicly available dataset.
There is no rationale given in section C.2 SYNTHETIC DATA GENERATION - on the choice of selected form, For instance X1 = 0.8Z1 + 0.4Z2 + 0.2Z3 + ϵ1. How does the approach work/scale in the presence of several latent variables (much more than 3) ?
The authors do no validate intent on a setting with ground truth or with review of high-q unlabeled entries done by health personnel.
The paper misses an ablation that replaces PAN with (a) a plain classifier of S̃ on a, and (b) nnPU with an estimated prior, keeping everything else fixed. In general the choice of PAN is not justified and there is no evidence provided that it works in this setup. In the synthetic study, where the true intent S is known, the quality of PAN's scores (AUC or calibration against S) is never reported.
Questions for authors
Why do the results not include confidence intervals ( Tables 2–5 ) ? How does the approach work/scale in the presence of several latent variables (much more than 3) ? Can the authors demonstrate the solution on a public real world dataset ? Can the authors somehow discuss what happens if the underlying latent model is more complicated than the assumed one ? Can the authors discuss why PAN is used ? MaybeA a plain classifier would likely do also. Under the standard assumption that labeled positives are a random sample of all positives an ordinary classifier might do the job. That classifier needs no prior, no adversarial training, and costs 167 logistic regressions or GBMs instead of 167 GAN-style pairs. The paper might try to assess this alternative.
11 comments
Response to Reviewer 38um (Q4)
Q4: Robustness to more complex latent models
Can the authors somehow discuss what happens if the underlying latent model is more complicated than the assumed one ?
We thank you for this interesting and constructive question.
If your concern is about the dimensionality of the latent variables, please refer to our responses to W3 and Q2, where we provide additional simulations with 16 latent variables and discuss the architecture used in our real-world experiments.
If you are referring to more complex generative models beyond VAE, our PU integration framework is not restricted to a specific generative architecture. The PU step estimates measurement-intent probabilities independently of the generative model. These probabilities can potentially be integrated into other architectures, such as diffusion-based or Transformer-based models, through conditioning and auxiliary supervision. For example, a diffusion-based model could condition its denoising network on :
Similarly, a Transformer-based model could integrate into its input representations:
We choose VAE because it allows us to jointly model observed values and missingness masks under MNAR. Its encoder, data decoder, and missingness decoder can be conditioned on , while the variational training objective allows us to add weighted auxiliary PU supervision. This makes VAE a natural choice for evaluating the contribution of measurement-intent information to both imputation and downstream prediction.
However, such extensions would require architecture-specific designs and additional evaluation. Given recent advances in diffusion-based imputation for tabular data, we consider this a promising direction for future work.
We have also discussed this possibility in the revised Conclusion:
Future work may extend the proposed framework to accommodate more complex missingness mechanisms through additional conditioning strategies and alternative generative architectures, including integration into diffusion-based and Transformer-based models.
Response to Reviewer 38um (Q3)
Q3: Evaluation on a public real-world dataset
Can the authors demonstrate the solution on a public real world dataset?
We appreciate your suggestion. As discussed in our response to W2, we use the Cerner Health Facts dataset to illustrate the application of PU-VAE-MNAR to real-world EHR data. Our framework is not specific to Cerner and can be adapted to other datasets with partially observed measurement-intent information.
We agree that evaluation on a public real-world dataset would improve reproducibility and provide further evidence of generalizability. However, the current study does not include such an experiment. Instead, we evaluate the framework using three simulation scenarios with known data-generating mechanisms and a large real-world EHR cohort of 106,261 patients.
The simulation studies allow us to evaluate imputation accuracy against known ground truth and examine the proposed method under different MNAR mechanisms. The Cerner experiments further evaluate its application under naturally occurring EHR missingness, using fixed patient-level training, validation, and held-out test partitions, as described in Appendix D.3.
We acknowledge the lack of public real-world validation as a limitation. Extending the evaluation to suitable public EHR datasets with identifiable measurement-intent information is an important direction for future work.
Response to Reviewer 38um (Q5)
Q5: Why is PAN preferred over conventional classifiers?
We sincerely appreciate your insightful suggestion. For additional experimental comparisons with conventional classifiers and nnPU, please refer to our detailed response to W5.
Empirically and theoretically, we adopt Predictive Adversarial Networks (PAN) [1] for measurement-intent estimation because PAN does not require a known positive-class prior and demonstrates robust classification performance even when only a small fraction of positive examples are labeled. Its adversarial learning objective encourages the identification of latent positives (S=1,S=0) within the unlabeled set U while distinguishing them from true negatives (S=0,S=0), aligning with our partially observed measurement-intent setting.
To further validate intent estimation, we have added expected calibration error (ECE) results against the ground-truth S in the right-hand table of Figure 2, providing an absolute assessment of probability calibration. The low ECE values indicate good calibration of the estimated measurement-intent probabilities. Furthermore, our real-world EHR ablation study (Table 6) demonstrates that PAN-based conditioning and PU supervision both improve imputation accuracy compared with nnPU. Specifically, RMSE decreases from 0.183 (nnPU) to 0.121 (PAN conditioning) and further to 0.110 (full PU-VAE-MNAR), representing a 39.9% reduction relative to nnPU. These findings support the empirical suitability of PAN for the proposed framework.
References:
[1] Wenpeng Hu, Ran Le, Bing Liu, Feng Ji, Jinwen Ma, Dongyan Zhao, and Rui Yan. Predictive Adversarial Learning from Positive and Unlabeled Data. Proceedings of the AAAI Conference on Artificial Intelligence, 35(9):7806–7814, 2021.