- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ProFIT addresses entity-disjoint few-shot forecasting of irregularly sampled multivariate time series: given only K asynchronous observation events (t, v, x) from an unseen entity (station/patient/subject), forecast values in a future window. The method learns a complete-task function latent via a variational encoder-decoder (attention pooling over event embeddings with multi-scale log-Fourier time features; FiLM-modulated Gaussian decoder), clusters standardized posterior means into P prototypes, and at test time routes the K-event context to a convex combination of prototypes using both a context embedding and explicit sampling statistics (span, gap moments, variable frequencies). A conditional latent diffusion prior (with time-progressive weakening of prototype guidance) generates task latents, trained with a forecast-alignment objective (predictive KL between teacher and generated latents) plus prototype anchor/usage-balancing losses.
Critical strengths
- A well-specified and practically relevant problem formulation. Entity-disjoint transfer with K-shot counting events (not windows), nested support sets, and fixed target windows is a clean, reproducible protocol that faithfully captures the "newly deployed station/patient" scenario. The explicit statement that this evaluates transfer rather than controlled sampling-shift interventions is honest scoping.
- Sampling-aware routing is a sensible and somewhat novel touch. Augmenting the context embedding with explicit sampling statistics (valid ratio, gap mean/std, variable frequencies) so that "similar-valued contexts with different spans, gaps, or coverage" route differently is a simple idea that directly targets a real failure mode of prototype/meta-learning approaches on irregular data.
- Efficiency reporting is exemplary for this venue's standards. Table 4 reports parameters, memory, latency with std over repetitions, and DDIM step–accuracy trade-offs, including the honest observation that non-diffusion baselines are faster but less accurate. Many comparable papers omit this entirely.
- Complete-task latent as teacher with diffusion prior at test time is a coherent way to get a proper posterior over task dynamics under extreme sparsity, and the forecast-alignment loss (KL on predictive distributions rather than latent vectors) is the right alignment target.
Critical weaknesses
- All results are single-run point estimates. The paper states this explicitly. Few-shot evaluation is notoriously high-variance, and several margins are small (e.g., Activity K=25: 1.591 vs. ASTGI 1.690; PhysioNet K=30: 0.964 vs. Hi-Patch 0.912). Without multi-seed error bars, the headline "lowest error on USHCN and Activity for every context budget" cannot be statistically assessed. This is the most consequential weakness.
- The PhysioNet claim is overstated relative to Table 1c. Hi-Patch beats ProFIT at K=5 (0.984 vs. 1.145) and at K=30 (0.912 vs. 0.964) in MSE. "Competitive" is technically defensible, but the abstract's framing (best on two datasets, "competitive" on the third) hides that ProFIT loses 2 of 4 K-settings on PhysioNet to a simpler non-generative baseline.
- Baseline adaptations look fragile. ST-GFSL and ASTGI produce catastrophically bad numbers at small K (USHCN K=1: 81.9 and 82.1 MSE, ~14× ProFIT), suggesting the "native interface" adaptation of these methods to event-level irregular few-shot input is broken rather than merely weak. Similarly, how UniTS (a regular-grid model) consumes asynchronous events "without interpolation or forward filling" is not specified. If baselines are handicapped by the interface, the margins are hard to interpret.
Questions for authors
See weaknesses above
Responses to Weakness 1: We thank the reviewer for pointing out the insufficiency of single‑run point estimates. We have supplemented experiments with model seeds 42, 43, and 44 for USHCN, Activity, and PhysioNet, reporting the mean ± sample standard deviation of normalized MSE, RMSE, and MAE for each K. The results and related discussion are added in Table 12, Appendix J.
Responses to Weakness 2: This paper studies the few‑shot forecasting problem for irregular multivariate time series on new tasks with inherent correlations. Strictly speaking, the PhysioNet dataset is not particularly suitable for our model, or rather, not entirely aligned with our problem. ProFIT's design goal is to leverage accumulated experience from training tasks to help unseen tasks with only a few irregular observations build predictive representations. The "unseen tasks" require inherent correlation with training tasks. For example, in the traffic sensor scenario, the newly added sensors and original sensors are in the area, and traffic flows are correlated. Thus, even if the target task lacks sufficient history, the model can forecast using reusable variation patterns from other tasks. USHCN weather station network and Activity human motion sensor network fit our definition. However, patient heterogeneity in PhysioNet causes a lack of this inherent correlation. Given limited availability of public irregular time‑series datasets, and our model's results being reasonable, we included it in the experiments. We will modify inappropriate performance descriptions like "competitive" in the revised manuscript to express our model's experimental results more appropriately.
Responses to Weakness 3: It should be noted that the catastrophic errors of ST‑GFSL and ASTGI at K=1 are not artifacts of broken interfaces, but reflect their architectural incompatibility with extreme event‑level sparsity. ST‑GFSL learns task‑specific transferable metrics by constructing spatio‑temporal graphs. When K=1, there are not enough nodes and edges to build any meaningful graph, causing metric learning to degenerate. Similarly, ASTGI relies on adaptive spatio‑temporal graph interactions. With only one asynchronous event across variables, it cannot compute cross‑variable dependencies, and its aggregation mechanism collapses. Regarding input for regular‑grid models like UniTS, introducing extra interpolation or padding would alter the original model and compromise fairness. Therefore, we use no interpolation or forward filling; instead, each asynchronous event (ti,vi,xi) is mapped to a continuous‑time input token. Specifically, UniTS receives an ordered sequence containing observed value xi, variable identity mask vi, and exact elapsed time Δti=ti−ti−1. This time‑aware tokenization is a standard and fair way for regular‑grid models to process irregular sequences without violating actual temporal gaps.
Therefore, the margins are fair and interpretable. Baselines specifically designed for continuous‑time or irregular data (e.g., APN, TimeFlow, Hi‑Patch) do not collapse at K=1; instead, they degrade gracefully. This proves our event‑level interface is fair and functional. To further validate the comparison, we have supplemented additional experiments in Table 1 with recent works adapted for irregular forecasting (Rubato, LightGTS) and state‑of‑the‑art general foundation models (TOTO 2, Moirai) under the same fair interface, proving the superiority of our work.