- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes MERL for longitudinal prediction from multimodal physiological signals. MERL builds on mixedML [1]. A CNN-Transformer learns the population-level function from biosignals and scalar covariates. A linear mixed model with a random intercept and three random slopes captures participant-specific deviations. The two parts are fit by alternating estimation. On simulated data, MERL achieves lower RMSE than the same network with or without one-hot participant identifiers when both fixed and random effects are present. MERL also recovers the ordering of the true variance components.
[1] Kilian et al., Mixed effects in machine learning: A flexible mixedML framework to add random effects to supervised machine learning regression, TMLR 2023.
Critical strengths
-
The paper clearly separates within-assessment temporal structure from across-assessment participant variation.
-
The simulation provides ground-truth variance components, so the authors can check the random-effects estimates.
-
The fixed-only and random-only outcomes serve as useful sanity checks.
Critical weaknesses
-
Limited novelty. mixedML [1] is already architecture-agnostic. MERL applies it with a CNN-Transformer and does not add a new estimation method or theory.
-
The simulation matches the model. MERL uses the same random-effects design that generates the data. The population function is a linear sum of features. The paper justifies only the feature correlations, not the variances, noise level, or sample size. It also does not test misspecification.
-
Missing baselines. NN and are ablations, not competing methods. The authors cite MeNets [2], LMMNN [3, 4], DeepGLMM [5], and Mandel et al. [6], which I appreciate. MeNets, LMMNN, and Mandel et al. apply directly to this setting with the same encoder. The paper notes that these methods couple the network and random effects differently. This difference is a reason to compare against them.
-
The biosignal part is not personalized. Only , , and have participant-specific slopes. The biosignals enter only the shared network. MERL therefore cannot model participants who respond differently to signal-derived quantities such as heart rate.
[1] Kilian et al., Mixed effects in machine learning: A flexible mixedML framework to add random effects to supervised machine learning regression, TMLR 2023.
[2] Xiong et al., Mixed effects neural networks (MeNets) with applications to gaze estimation, CVPR 2019.
[3] Simchoni and Rosset, Using random effects to account for high-cardinality categorical features and repeated measures in deep neural networks, NeurIPS 2021.
[4] Simchoni and Rosset, Integrating random effects in deep neural networks, JMLR 2023.
[5] Tran et al., Bayesian deep net GLM and GLMM, Journal of Computational and Graphical Statistics 2020.
[6] Mandel et al., Neural networks for clustered and longitudinal data using mixed effects models, Biometrics 2023.
Questions for authors
-
Table 1 seems inconsistent with Equations 14 to 16. The random effects have mean zero and are independent of the features, so . Under equal variances, the global-mean RMSEs give and . The mixed global-mean RMSE should then be near 9.6, but Table 1 reports 11.210. In the heterogeneous setting, the implied rises from about 1.9 to about 43, although no variance increases by more than a factor of four. Can the authors report the empirical variances of and and explain this?
-
Why are the variance components underestimated in Table 2? For example, the residual variance in the "Mixed" column is estimated at 0.792 and 0.817 under the equal and heterogeneous settings, while the true value is 1.
-
How should practitioners choose the random-effects design on real data? How sensitive is MERL to this choice?
-
MERL models participant-specific effects with a linear mixed model. Could the framework replace it with a more expressive nonlinear model, such as a shallow neural network over participant identifiers and participant-level features?