- Rating
- 2
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes chirp-modal Gaussian mixtures for probabilistic forecasting of irregularly sampled time series, which extend fixed-pole modal representations by adopting time-varying frequency and decay. It also retains analytic state transitions and temporal covariance with two strategies: a direct one-pass mixture predictor (CMD-X) and a diffusion-based iterative predictor (CMD-D).
Empirical evaluations span four benchmarks, showing improvements in joint forecast metrics, energy score, and variogram score over fixed-pole variants and competitive baselines. Further controlled ablations, distributional analysis, and theoretical derivations support the empirical results.
Critical strengths
- . The combination of time-varying modal dynamics and explicit temporal covariance offers a principled way to model more natural future trajectories at irregular query times.
- . The experiments cover four datasets, multiple baselines, and 10 training seeds. Capacity-matched fixed-pole comparisons and controlled interventions help isolate the contributions of the claimed components.
- . The direct and diffusion-based variants offer different quality–efficiency trade-offs.
Critical weaknesses
- The Gaussian components assume independence across latent coordinates, which may restrict their ability to represent complex multivariate dependencies. Although cross-coordinate dependence can emerge through mixture assignments and nonlinear decoding, the paper does not empirically establish whether these mechanisms are sufficient.
- . The paper could handle the distinction from LLapDiff more carefully. Although LLapDiff utilizes time-invariant modal poles within each denoiser evaluation, these poles are already conditioned on the observation history and noisy latent state, allowing them to adapt across contexts and diffusion steps. Therefore, CMD's contribution is precisely introducing within-window temporal variation in frequency and decay, rather than adaptive modal parameterization itself.
- The covariance construction is guaranteed to be positive semidefinite on each requested grid, but the query-grid numerical approximation does not generally guarantee consistency across different query sets. Although the appendix provides a fixed-anchor alternative, its empirical implications remain unexplored.
- While the paper reports parameter counts and inference latency, a unified performance–latency Pareto analysis (plot) is missing, especially against external forecasting baselines.
Questions for authors
-
The component-pool evaluation retained components that consequently contribute no samples, which makes it difficult to isolate the effect of mixture size from component ordering and sampling allocation. Please provide a cleaner comparison or give a self-consistent clarification.
-
Analysis of marginal calibration across forecasting horizons seems limited; e.g., some results indicate noticeable undercoverage. For instance, on NOAA-UK, the empirical coverage of nominal 80% intervals is 62.4% for CMD-X and 73.1% for CMD-D. Please justify whether improvements in joint scores consistently translate into reliable marginal forecasts.
-
How would the authors reposition and scope the paper given that the novelty differentiation is not sufficient, and perhaps a more precise comparison with competitive similar baselines would strengthen the contribution's positioning?
3 comments
4 Quality and latency
Figure 9 in Appendix I now plots mean ES against reported latency for CMD and external baselines on USHCN and NOAA-UK. Both CMD variants are nondominated among the displayed configurations: CMD-X takes approximately 19–20 ms/window, while CMD-D takes 128–146 ms/window and attains lower mean ES. In these plotted settings, CMD-X also has lower ES and reported latency than LLapDiff and Chirp-MSE. Faster external methods occupy other frontier points. The plot makes the practical choice between direct and iterative generation explicit for these configurations.
5 Component pool allocation and the main results
To address your concern, we added a controlled comparison separating sample allocation from retained-component selection, using existing trained models and a fixed budget of 25 scored paths (Appendix D.1).
Cleaner comparison. We compare A, the original prefix/ordered allocation; B, balanced allocation over the same prefix; and C, balanced random code subsets for CMD-X. Thus B−A tests allocation at fixed membership, while C−B tests membership under balanced allocation. Every retained component contributes under B/C: at M=8, all eight receive 3–4 paths, whereas A uses only seven.
Results. At M=8, balancing lowers CMD-D mean ES by 1.49% on USHCN and 2.51% on PhysioNet, with VS also improving (Table 11). CMD-X mean ES decreases but VS increases. Subset effects depend on pool size and metric: random subsets lower PhysioNet CMD-X mean ES by 2.80% at M=8, but increase it at M=16 (Table 12). These controls directly quantify allocation and selection sensitivity.
Reliability and scope. We use five checkpoints per model and dataset, averaging five matched evaluation repetitions within each checkpoint before reporting paired changes and across-checkpoint SDs. Table 13 separately reports evaluation variability. At the primary M=S=25 setting in Tables 1–3, every component supplies one path and the allocation protocols coincide, so the primary comparisons have no unused components. We therefore interpret intermediate curves as inference-time retained-pool operating points, rather than isolated effects of training mixture size.
6 Marginal calibration in context
Thank you for drawing attention to this distinction. We agree that better joint scores do not guarantee nominal marginal coverage, and that the NOAA-UK results reveal a remaining calibration limitation. For nominal 80% intervals, CMD-X and CMD-D achieve empirical coverage of 62.4% and 73.1%, respectively, both below the target.
The full results provide additional context. Empirical coverage of nominal 80% intervals is 79.0%/82.0% on PhysioNet and 74.4%/76.2% on USHCN for CMD-X/CMD-D (Tables 17–18). On NOAA-UK, CMD-D’s 73.1% coverage improves substantially over 53.5% for LLapDiff and 51.6% for ProFITi, alongside lower CRPS (Table 20). These comparisons show improvements in forecast scores and coverage relative to those baselines, while leaving residual undercoverage.
Calibration is also not uniformly strongest across methods: TFMixer achieves 73.6% coverage and lower PIT-ECE on NOAA-UK, despite worse ES and CRPS. We therefore distinguish improvements in predictive scores from evidence of calibrated uncertainty.
We recognize that these aggregate diagnostics do not fully address your concern about calibration across forecasting horizons: pooled coverage can conceal horizon-specific deviations. Appendix E reports coverage together with interval width, but a horizon-resolved evaluation of these quantities remains an important validation step. We have explicitly acknowledged the remaining calibration limitation and scoped our claims to the demonstrated forecast-score improvements, without presenting them as proof of nominal coverage across horizons.
We hope these targeted experiments, theoretical characterization, and clearer scope address your concerns and support a more favorable assessment of the revised paper.
Re: Official Comment by Authors
Thank you for the response and the substantial additional experiments and analyses. The revision addresses most of my original concerns and improves the paper.
In particular, the distinction from LLapDiff is now much clearer, and the capacity-matched fixed-pole comparisons provide more convincing evidence for the contribution of within-window frequency and decay variation. The additional cross-coordinate dependence experiment, fixed-anchor consistency analysis, and component-allocation controls also strengthen the methodological and empirical evaluation. The quality–latency comparison further clarifies the practical tradeoff between CMD-X and CMD-D.
A few limitations remain. First, the cross-coordinate dependence experiment demonstrates useful but incomplete recovery under a controlled synthetic setting, without establishing its sufficiency on real datasets. Second, the fixed-anchor consistency guarantee applies to CMD-X under specific conditions and does not extend to CMD-D; numerical discrepancies under TF32 also remain. Third, marginal calibration, particularly across different forecasting horizons, remains insufficiently characterized, although the authors now acknowledge this limitation and appropriately scope their claims.
Overall, I appreciate the authors' substantial efforts. I am therefore inclined to revise my original assessment more favorably, yet there is not an option between 2 and 3. Although the rating is not changed, I am inclined towards a borderline instead of a solid 3.
Thank you for recognizing the formulation and controlled evaluation, and for identifying the analyses needed to strengthen the paper. We have added dependence and consistency diagnostics, clarified the component-pool experiment, and included the requested quality–latency plot.
1 Empirical and theoretical analysis of latent independence
Thank you for highlighting the need to examine whether within-component latent independence limits the model’s ability to capture multivariate dependence. In response, we have added a controlled empirical study and a theoretical characterization of the benefits and limitations of shared mixture assignment.
Controlled empirical evidence. Appendix H.4 tests shared mixture assignment directly using two-coordinate trajectories and identity encoder/decoder maps. A same-checkpoint product control preserves both full coordinate-trajectory marginals while removing cross-coordinate coupling. Retaining the learned coupling improves the joint log score by 0.0333 and 0.0941 nats/value at correlations 0.5 and 0.9, respectively (Table 28). Separately, comparing models trained with one and 25 components shows reductions in cross-covariance RMSE from 0.2523 to 0.1121 and 0.4541 to 0.2077, respectively (Table 29). These complementary comparisons demonstrate that shared mixture assignment can recover useful cross-coordinate dependence despite independence within individual components.
Theoretical characterization. Equation 54 gives a finite-mixture lower bound on population excess KL. At correlation 0.9, ten queries, and 25 components, the bound is 0.254 nats/value, compared with the fitted model’s empirical excess NLL of 0.332 nats/value. Thus, alongside the demonstrated dependence recovery, the analysis identifies a structural limitation of coordinate-factorized mixtures at a fixed context before decoding.
We agree that these controlled results do not establish whether shared mixture assignment and learned decoding adequately capture dependence on real datasets. Appendix A.5 additionally derives a shared-state covariance extension that permits within-component cross-coordinate dependence, providing a concrete formulation for further investigation. A controlled real-data comparison with this extension, using matched representations and training/evaluation settings, remains an important validation step. We acknowledge this remaining limitation and explicitly scope the conclusions in Appendices H.4 and M.
2 Precise contribution relative to LLapDiff
We agree with your distinction and now state it explicitly in "Introduction" and "Related work" sections: LLapDiff already adapts modal parameters to history, noisy latent state, and diffusion level. CMD introduces within-window variation in physical time, coupled to both the mean and temporal covariance.
CMD-X is the primary realization; CMD-D studies iterative generation of the same family. The controlled comparison is against fixed-pole direct mixtures sharing the representation, likelihood objective, and covariance construction. CMD-X lowers mean ES by 4.0–27.9% across all four datasets against closely capacity-matched controls (Table 1; Appendix J). This provides evidence for the precisely scoped contribution.
3 Fixed anchor consistency in practice
To address your concern, Appendix B.2 now evaluates fixed-anchor (FA) inference on five existing CMD-X checkpoints per dataset on PhysioNet and USHCN. We use 64 anchor intervals, retain the learned weights and decoder, and compare matched samples without retraining.
Cross-query consistency. We test subsets, off-anchor supersets, permutations and irregular close-pair grids, with 8,000 component cases per comparison per dataset. Maximum relative covariance discrepancies under the original query-grid construction reach 0.126 on PhysioNet and 0.587 on USHCN. FA reduces these to zero observed discrepancy in float64 and at most 1.74 × 10⁻⁶ in float32 with TF32 disabled, meeting the declared tolerances (Table 4).
Effect on forecasting. In a separate TF32-enabled evaluation, absolute mean ES/VS/CRPS changes across five paired checkpoints are below 0.02% of the original ten-seed reference means (Table 5). Repeated sampling gives similarly small aggregate changes. TF32-enabled consistency errors reach 1.45 × 10⁻³ and exceed the float32 tolerance, so predictive effects and numerical consistency are reported separately.
Guarantee and scope. Corollary 1 establishes cross-query consistency with history-only component parameters and weights, query-independent anchors and time conventions, and pointwise decoding. This covers CMD-X under these conditions; CMD-D’s query-dependent generator remains outside the guarantee. The main forecasting tables retain the original query-grid construction. The revision thus adds empirical consistency tests and a controlled assessment of predictive effects using existing checkpoints.