- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Neural Belief-State Filtering (NBSF), a continuous-discrete filtering framework for irregular multivariate time series. Instead of maintaining only a deterministic hidden state, NBSF represents the evolving state as a probabilistic belief and separates temporal propagation from observation-time assimilation. Its main contribution is an event-time cross-variable update, allowing an observation of one variable to immediately revise beliefs about other unobserved variables without maintaining a dense covariance matrix. The paper evaluates local-coordinate, dense-gain, and nonlinear cross-variable updates across several clinical datasets and a controlled sensing dataset, with the main finding that cross-variable assimilation is particularly useful when direct observations of the target variable are scarce.
Critical strengths
The strongest part of the paper is the mechanism-level evaluation. The authors do not rely only on overall classification performance; they test belief quality before direct observation, progressively remove target observations, compare against a parameter-matched wide-local model, and also destroy cross-variable structure by permuting non-target variables across patients. The latter is especially convincing because the advantage of the cross-variable models disappears and eventually reverses as cross-variable information is destroyed, which directly supports the claimed mechanism rather than merely showing a larger model performs better.
Critical weaknesses
My main reservation is that the strongest evidence supports event-time cross-variable information transfer, but not necessarily the full nonlinear NBSF update proposed as the main model. In several experiments, the much simpler dense-gain variant is as good as or better than the nonlinear cross-variable update. Dense-gain gives the lowest CRPS in all four core belief-quality comparisons, performs more consistently under target suppression, and is best in most subsystem-holdout settings. In addition, the propagation ablation shows that the hybrid propagation mechanism is not consistently better than the simpler SSM-only or MLP alternatives, and removing the latent block actually slightly improves P19 classification. These results do not weaken the central cross-variable-assimilation idea, but they do suggest that some architectural components of the full NBSF formulation are less strongly justified than the paper’s broader method presentation may imply.
Questions for authors
The main question I would like the authors to clarify is what they consider the essential contribution of NBSF after the ablation results. The experiments seem to provide strong evidence for event-time cross-variable assimilation, but considerably weaker evidence that the nonlinear update, latent state, or hybrid propagation are necessary. Since dense-gain is frequently the strongest variant, would the authors view the simpler dense-gain filter as the preferred realization of the proposed principle? I think making this distinction clearer would strengthen the paper, because the empirical contribution appears to be more about where cross-variable information enters the filtering process than about the complexity of the neural update itself.
4 comments
Dear Reviewer,
We would be grateful if you could kindly revisit our earlier author response, where we addressed your important questions regarding the essential contribution of NBSF and the role of its architectural components.
Your observations have helped us clarify that the central contribution lies in event-time cross-variable information transfer, rather than the complexity of the nonlinear update.
We have also completed several additional controlled experiments to further substantiate this interpretation. We respectfully present these findings in the following comment for your consideration.
Following our earlier response, we conducted additional controlled experiments to strengthen the evidence for event-time cross-variable assimilation and establish whether the simpler dense-gain update is the preferred realization.
1. Essential contribution and preferred realization
Our central contribution is event-time cross-variable assimilation within a predictive belief-state filter. We agree with your interpretation that the simpler dense-gain update is the preferred realization in settings where this mechanism is beneficial. The nonlinear update, latent block, and hybrid propagation are optional architectural choices, consistent with Appendices B.5–B.6.
2. Fixed observation projection and robustness to missing observations
To isolate the assimilation mechanism from a learned observation projection, we retrained the models with
verified frozen in every checkpoint.
We evaluated test-time removal of observations across all variables in the first 25% of each stay, using identical masks across models.
Dense-gain minus local AUROC (paired 95% CI; favorable folds):
| Dataset | No removal | 90% removal |
|---|---|---|
| P12 | +0.0148 [+0.0080, +0.0217], 5/5 | +0.0787 [+0.0681, +0.0891], 5/5 |
| MIMIC-III, ICU stays h | −0.0005 [−0.0050, +0.0039], 3/5 | +0.0440 [+0.0378, +0.0507], 5/5 |
| P19 | −0.0014 [−0.0069, +0.0041], 2/5 | +0.0037 [−0.0014, +0.0090], 4/5 |
The advantage increases substantially under early-observation removal on P12 and MIMIC-III. The MIMIC-III eligible-stay results supersede our earlier all-stays figures. P19 shows no clear improvement.
3. Parameter-matched and multi-seed experiments
We matched the local-coordinate model's capacity to dense-gain (122,684 versus 122,671 parameters).
On P12 at 90% removal, dense-gain exceeds the parameter-matched wide-local model by
\quad 95\%\ \mathrm{CI}=[+0.0994,+0.1227].$$ The advantage persists across three training seeds: | Seed | Dense-gain − local AUROC, 90% removal | |---|---| | 42 | **+0.0787** [+0.0681, +0.0891] | | 43 | **+0.0547** [+0.0450, +0.0643] | | 44 | **+0.0570** [+0.0471, +0.0670] | All five folds favor dense-gain under each seed. These experiments use the same patient folds with different training initializations. Extending the originally capped training runs by up to 200 additional epochs leaves the improvement nearly unchanged. **4. Additional evidence for cross-variable information transfer** We specified a variable-permutation intervention and its success criterion before execution. Observed variable values were independently shuffled across test cells while preserving timestamps and observation masks. At 90% removal, the dense-gain advantage changes from: - **P12:** +0.0787 to −0.0605, a reduction of **0.1392** [0.1244, 0.1535]. - **MIMIC-III:** +0.0440 to −0.0354, a reduction of **0.0794** [0.0705, 0.0885]. Both reductions are observed across all five folds. This provides further evidence that the model uses information in relationships among variables, although the permutation also introduces distribution shift. **5. Full-covariance Kalman comparison** We additionally implemented a full-covariance Kalman-style update with the same propagation network and fixed observation projection. On P12 at 90% removal: - **Dense-gain versus local:** +0.0787 [+0.0681, +0.0891]. - **Full covariance versus local:** +0.0135 [+0.0036, +0.0237]. Full covariance requires approximately **2.0 times the training time and 8.4 times the peak GPU memory** at batch size 64. Although it achieves better one-step-ahead CRPS, it does not reproduce the magnitude of the dense-gain classification improvement. **6. Further clarification of the architecture** On P12 without removal, learned-projection dense-gain and nonlinear achieve similar AUROC (0.8438 and 0.8467). The dense-gain result supersedes the previously quoted 0.8312, which used a different variance-update implementation. The earlier eICU all-stays figures are likewise superseded: on unit stays of at least 48 hours, local-coordinate achieves 0.776, the cross-variable variant 0.771, and dense-gain 0.765 AUROC. These additional experiments reinforce the conclusion that the principal contribution is **where and when cross-variable information enters the filtering process**. Dense-gain provides a simpler realization of this mechanism, with robust classification improvements under missing early observations on P12 and MIMIC-III. We sincerely thank you for your thoughtful feedback and careful evaluation. We hope you will consider our earlier response together with these additional experimental findings when making your final assessment.Thank you to the authors for their detailed responses and additional experiments. I appreciate their efforts to clarify the central contribution of NBSF and to provide further evidence supporting event-time cross-variable assimilation, particularly under substantial observation missingness.
The new results help address some of my earlier concerns and strengthen the empirical support for the proposed mechanism. However, they also reinforce my original assessment that the main contribution lies in cross-variable information transfer rather than the specific architectural components of the full NBSF model. Moreover, the improvements remain dependent on the dataset and experimental setting, with less consistent benefits on P19 and eICU.
While I acknowledge the value of these clarifications and additional experiments, I do not find them sufficient to change my overall assessment.
We thank the reviewer for recognizing the mechanism-level experiments. We agree that NBSF’s essential filtering principle should be distinguished from optional architectural complexity.
1. What is the essential contribution?
NBSF propagates a factorized Gaussian belief across irregular gaps and assimilates measurements through learned, innovation-driven corrections, with O(S) filtering-state storage. The main question is when an observation should immediately revise beliefs about unobserved variables.
Cross-variable updates already exist in Kalman filtering; we do not claim otherwise. Our contribution is a learned continuous-discrete realization and controlled tests of the jointly learned observation representation and assimilation mechanism. Local, dense-gain, and nonlinear variants share a common framework. Parameter-matched controls, before-first and pre-assimilation prediction, target suppression, subsystem holdouts, and information-destruction tests examine the mechanism.
2. Is dense-gain the preferred realization?
In many evaluated settings, yes. Dense-gain is a simpler and frequently more effective default. In Appendix B.7 (Table 15), it achieves the lowest CRPS among the three standard NBSF variants in all four core belief-quality settings:
Dense-gain is also best in 7/10 subsystem-holdout settings (B.14, Table 25), and is more consistent under target suppression (B.12, Table 22). Hence where information enters the filter can matter more than update complexity.
However, nonlinear assimilation remains useful in some tasks: on P12 mortality it yields AUROC/AUPRC 0.8467/0.5160 versus 0.8312/0.4906 for dense-gain (Table 12), and on patient-grouped eICU it yields 0.8844/0.5791 versus 0.8788/0.5661 (Appendix B.3, Table 11). Thus, dense-gain is our preferred simple default for the tested learned-projection settings, not a universally optimal variant.
3. Are hybrid propagation and latent dimensions required?
No; the ablations do not establish that requirement. On P19, SSM-only propagation achieves AUROC/AUPRC 0.9312/0.6317, compared with 0.9211/0.6117 for hybrid propagation. On P12, their AUROCs are almost identical (0.8465 versus 0.8467; Appendix B.5, Table 13). Similarly, removing the latent block (L=0) increases P19 AUROC from 0.9211 to 0.9249 (Appendix B.6, Table 14). We therefore treat the hybrid residual and latent block as optional capacity choices, not prerequisites for the claimed observation-assimilation mechanism. The results do not justify presenting the complete nonlinear architecture as intrinsically superior.
4. New control: does the update alone explain the gains?
To probe a possible confound, we additionally ran a five-fold P19 fixed-projection ablation. The standard system learns the observation projection Wobs jointly with the update. We replaced it with a fixed coordinate-selection projection, Wobs=[ID∣0]. Mean AUROCs were 0.9255 (local), 0.9241 (dense-gain), and 0.9005 (nonlinear). Under this control, neither cross-variable operator improves mean AUROC over local-coordinate.
This negative control prevents us from attributing learned-projection gains to the update operator in isolation. Our mechanism experiments support the narrower conclusion that jointly learned observation representation and assimilation can exploit cross-variable information, especially with sparse target observations. We will report the full fixed-projection protocol in the camera-ready version if accepted.
5. Clarification of the paper's contribution
We therefore distinguish (i) the probabilistic continuous-discrete belief representation and innovation-driven update, (ii) the empirical role of observation-time information transfer under missingness, and (iii) optional architectural choices. The first two are central to NBSF; the third is task-dependent. Dense-gain is often the more parsimonious implementation, while nonlinear assimilation, hybrid propagation, and latent capacity are alternatives whose benefits must be demonstrated rather than presumed. We will make this hierarchy explicit in the camera-ready presentation if accepted.