- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PAMF, a method that first imputes missing signals in multimodal time series before performing classification. For localized missing values, it leverages nearby observations; for cases where an entire modality is missing, it begins with the channel mean and incorporates existing signals, completing the imputation via conditional flow matching. To harness information from the classification task, the authors first train a classification encoder and copy its weights to the imputation encoder. Once the imputation model is trained, it is frozen, and the classifier is then trained to adapt to the imputed data.
Critical strengths
- The experiments have a broad scope. The paper is evaluated on four benchmarks with controlled missing data and two clinical datasets containing naturally occurring missing data, covering scenarios such as partial missingness, complete modality absence, mixed missingness, and high rates of missing data.
Critical weaknesses
-
The training and model-selection protocols for the main classification baselines are insufficiently documented, and the additional computational cost remains unclear. The paper provides a detailed account of the three-stage training configuration and checkpoint selection criteria for PAMF, but it fails to specify the corresponding training epochs, hyperparameter selection processes, and model selection criteria for key baselines such as Maestro, FuseMoE, Flex-MoE, and MIRA. Consequently, it is currently difficult to verify whether these methods were appropriately trained and tuned, which also hinders the reproducibility of the experiments. In addition, PAMF introduces an additional completion encoder, flow-matching module, and multi-stage training, but the computational efficiency comparison only includes the diffusion completion method and does not report its total training cost relative to the direct classification baseline and end-to-end inference latency. The absence of this information undermines the credibility of the primary performance comparisons.
-
Lacks comparison with closely related flow-based completion methods. The paper discusses flow-based imputation methods such as CLWF, MIRI, and CFMI in related work, but the direct completion experiment only compares CSDI and SSSD. Although the "Related Work" section outlines how these methods differ from PAMF in terms of their conceptual positioning, it fails to explicitly explain why certain methods were deemed unsuitable for the current evaluation framework or why they were excluded from the comparison. Given that CFMI and CLWF have already been evaluated for imputation tasks using multivariate clinical time-series data from PhysioNet, they should be considered relevant, representative baselines.
Questions for authors
-
What training and model selection protocols were used for the primary baselines? Please specify the number of training epochs (or stopping criteria), the hyperparameter selection process, and the checkpoint selection criteria for Maestro, FuseMoE, Flex-MoE, and MIRA. This information is essential for determining whether the reported gains are built upon baselines that have been sufficiently trained and reasonably tuned.
-
Why were flow-based completion methods with clinical time-series experience not included? The paper discusses CFMI and CLWF, both of which have been evaluated in experiments involving the imputation of multivariate clinical time series. Could you clarify why these methods were not included in the direct imputation comparison? Could you incorporate a suitable representative method into the matching data partitioning, preprocessing, missing masking, and downstream classification training protocol to compare reconstruction quality, classification performance, and inference cost?
-
How much additional end-to-end computational cost does PAMF incur compared to direct classification methods? Please report the total training time and end-to-end inference latency for PAMF versus direct-classification baseline, ensuring that both hardware and batch size are matched. These results will help determine the additional computational cost incurred for any gains in classification performance, rather than simply assessing whether PAMF is faster than the chosen diffusion-based completer.
-
What specific factors are altered by "No Prior" and "No Transfer" ablations? Please clarify the specific implementation of the two ablation studies shown in Figure 3. For "No Prior," what source state is used in place of the proposed initialization? For "No Transfer," are the pre-training of the prediction branch, the fusion encoder, the classification head initialization, and the subsequent training pipeline all kept the same, with only the copying of weights to the completion encoder omitted?
-
Do the normalization and source-state statistics rely solely on information available at inference time? For datasets that employ in-sample or in-window normalization, are the mean and standard deviation calculated before or after the application of the missing-value mask? Are the artificially hidden values included in the calculation of these statistics?
3 comments
Thank you for the detailed rebuttal and additional experiments. I still have a concern about normalization. In practice, the missing signal values would not have been measured, but normalizing the complete test signal before masking uses those values. Could this make the test results overly optimistic? Could you retrain and evaluate PAMF and a strong baseline on PTB-XL under mixed missingness, using normalization that does not rely on hidden values, to see whether PAMF’s advantage remains?
Thank you for raising this concern.
PAMF's advantage over MAESTRO remains when normalization uses only training-set statistics: PAMF achieves higher Macro-F1 for all three seeds, with an average gain of 3.17 percentage points.
We retrained and evaluated PAMF and MAESTRO on PTB-XL under 20% mixed missingness, applying the missingness mask before normalization and using statistics computed only from the training set. Thus, masked test values do not contribute to the normalization of test inputs.
| Method | Macro-F1 ↑ | AUROC ↑ |
|---|---|---|
| PAMF | 0.637 ± 0.005 | 0.869 ± 0.009 |
| MAESTRO | 0.605 ± 0.024 | 0.866 ± 0.005 |
Results are mean ± sample standard deviation over seeds 42, 52, and 62. These results show that PAMF's Macro-F1 advantage persists under this normalization protocol.
Dear D3rU,
We thank you for your careful review and for recognizing the broad range of missingness settings in our experiments. We address your concerns and questions below.
W1/Q1: Baseline training and model selection
We retain the baselines’ core fusion mechanisms, report their training configurations, and select checkpoints by validation Macro-F1. All methods use an 80% / 10% / 10% training / validation / test split. The baseline configurations are listed in our response to Reviewer icex (W1/W4/Q1). We have clarified the baseline hyperparameter settings in the Experiments section of the revised manuscript.
For PAMF, as stated in the appendix, we select the imputation checkpoint by the lowest validation RMSE and the final prediction checkpoint by the highest validation F1.
W1/Q3: Training and inference cost
Compared with direct classification using the same backbone, PAMF raises Macro-F1 on PTB-XL from 0.605 to 0.641, at about 3.3 times the training time and 4.2 times the inference latency. On PTB-XL under 20% mixed missingness, No Imputation uses the same classification backbone to classify incomplete inputs directly. It achieves Macro-F1 of 0.605 ± 0.022, while PAMF achieves 0.641 ± 0.011.
Training times are summarized over seeds 42, 52, and 62. Inference latency was measured on an NVIDIA A6000 GPU with a batch size of 64.
W2/Q2: Comparison with a flow-based imputer
PAMF achieves lower reconstruction MSE (0.477 vs. 0.690) and higher Macro-F1 (0.641 vs. 0.634) than CFMI, a flow-matching imputer, under the same PTB-XL protocol. We include CFMI as an additional flow-matching baseline on PTB-XL under 20% mixed missingness. The table also includes the CSDI, SSSD, and PAMF results reported in Table 4.
Inference latency was measured on an NVIDIA A6000 GPU with a batch size of 64.
Although CFMI uses 20 function evaluations, its latency is higher in the reported measurements. One possible contributing factor is the cost of each network evaluation: the CFMI implementation uses four Transformer layers following Tashiro et al., each computing attention over both the temporal and lead dimensions.
Q4: Definitions of No Prior and No Transfer
No Prior uses a zero source, while No Transfer omits only the copying of pretrained prediction-encoder weights to the imputation encoder.
We have clarified these definitions in Section 4.5 of the revised manuscript.
Q5: Normalization and source-state statistics
On PTB-XL, each complete ECG window is normalized per lead before synthetic masking; this preprocessing is identical for PAMF and all baselines. PAMF's source prior itself uses only values that remain visible after masking.
After masking, PAMF constructs its source prior using visible values: the temporal prior uses visible neighboring observations, and the modality-level prior uses the corresponding channel statistics from samples in the current batch where that modality is observed.