acceptodds
Under review as a conference paper at ICLR 2027

Scale-Aware Foundation Priors for Irregular Multivariate Time-Series

Abstract

Irregular multivariate time series (IMTS) are widely encountered in sequential modeling, yet irregular sampling makes the effective temporal scale vary locally within a single series. However, most existing IMTS methods focus on representing asynchronous and missing observations, still lack explicit local scale adaptation, thereby overlooking this local scale heterogeneity. Foundation models already encode multi-scale temporal structure, but this prior remains locked to a regular sampling grid. To fill this gap, we propose ScaleIMTS, which adapts foundation priors to IMTS in a scale-aware manner. It first conditions these priors on local sampling scales by routing each temporal segment to scale-specific experts. The resulting representations are then refined by an IMTS-specific asynchronous structure: raw events are kept in time-aligned patches to capture local temporal patterns and time-varying inter-variable dependencies, and these features refine the already scale-adapted priors. The refined representations are fed into task-specific decoders to adapt to different downstream tasks. Experiments on 7 datasets demonstrate that ScaleIMTS outperforms state-of-the-art models in IMTS forecasting and classification tasks. Code is available at: https://anonymous.4open.science/r/scale_imts-62E0.

Primary area:
learning on time series and dynamical systems
Keywords:
Irregular Multivariate Time Series, Local Scale Heterogeneity, Time Series Forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3h5EOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

In order to account for the varying sampling density within one sample, the paper proposes ScaleIMTS, which mainly includes scale expert router, event patch encoder, and prior refinement module. Experiements on forecasting and classification tasks demonstrate its competitve performance.

Critical strengths
  1. The experimental comparisons are up-to-date and comprehensive.
Critical weaknesses
  1. "Scale" is used interchangably with "sampling density" in this paper, which makes it really unprofessional and weird. For example, the claim from line 14 "existing methods lack explicit local scale adaptation", should use "lack explicit sampling density considerations". Moreover, the claim is weak. For a non-multi-scale method, there's no "scale" concept in its network. Also, why "lack explicit local scale adaptation" will "overlook this local scale heterogeneity"? Even without any patch or multi-scale techniques, classic methods like Raindrop[1] already leverage the sampling density information in their learning.
  2. Model structure is redundant, and Figure 2 is horrible. Why would we need both a scale MoE branch and a patch encoder branch in parallel, now that they both encodes IMTS data? Why not sequential or merged as one? Why different experts are essentially of the same MLP structure instead of being truely scale-specific? Is prior refinement module a trick-only module?

[1] X. Zhang, M. Zeman, T. Tsiligkaridis, and M. Zitnik, “Graph-Guided Network for Irregularly Sampled Multivariate Time Series,” presented at the International Conference on Learning Representations, 2022.

Questions for authors

See weaknesses.

1 comment

Dear Reviewer 3h5E,

Thank you for the constructive review. We have clarified the terminology and contribution, explained the complementary roles of the model components, and added a controlled analysis of expert specialization.

Q1: Justification of “Scale” claims

We define local temporal scale as the effective temporal granularity within variable vv and window WW, measured by the mean inter-observation interval Δv,W\Delta_{v,W}. Sampling density describes observation frequency. The concepts are related—denser observations imply shorter intervals and finer observed granularity—but are not interchangeable. “Scale” describes the sampling pattern, not an assumed multi-scale architecture in every baseline.

Raindrop already models irregularity: it builds sample-specific sensor graphs and uses observation and time information to capture sample- and time-varying sensor dependencies, then aggregates observation representations over time. ScaleIMTS complements this by summarizing timestamps within each variable-specific segment and using these statistics to route frozen Timer representations among short-, mid-, and long-context experts. Its Event Patch Encoder represents each event by its value, timestamp, and preceding interval, and models temporal patterns across patches and cross-variable interactions within patches. ScaleIMTS thus explicitly adapts pretrained temporal representations to local interval changes.

Q2: Figure issue and clarification of the MoE branch

The Scale Expert Router and Event Patch Encoder process complementary representations. The router partitions frozen Timer tokens into segments and applies corrections conditioned on local timestamp statistics. The patch encoder processes raw events to represent asynchronous local dynamics and patch-specific cross-variable interactions. Its global irregular context informs the router, while its variable-level features condition prior refinement.

The three experts have independent parameters and distinct inputs despite sharing a two-layer MLP form: the short expert uses the last valid token, the mid expert uses within-segment variation, and the long expert uses the first token and element-wise maximum. The router mixes their corrections using segment statistics and context.

We tested specialization by disabling each expert at test time, renormalizing the remaining routing weights, and grouping samples by training-set interval tertiles. Entries show metric changes from the full model; bold indicates the largest degradation in each group.

DatasetIntervalOff-ShortOff-MidOff-Long
USHCNShort+0.071+0.058+0.004
Mid+0.026+0.092+0.063
Long+0.015+0.019+0.061
Human ActivityShort+0.0072+0.0010+0.0006
Mid+0.0011+0.0058+0.0034
Long+0.0007+0.0013+0.0047
P12Short−0.186−0.028−0.014
Mid−0.021−0.079−0.025
Long−0.012−0.026−0.099
P19Short−2.119−1.181−0.693
Mid−1.225−2.853−0.831
Long−0.940−2.312−3.114

Across all four datasets, removing the expert matched to an interval group produces that group’s largest metric change, showing distinct expert contributions across local sampling regimes.

Prior Refinement uses an MLP conditioned on the Event Patch feature and target-time query to generate per-variable affine scale and shift parameters for the scale-adapted prior. On P19, the full model obtains 93.4 AUROC/68.2 AUPRC, versus 90.8/65.4 with Additive Update and 90.7/63.1 with Concat Fusion; removing Event Patch lowers AUPRC to 64.4. These ablations show the value of query-conditioned refinement and event features.

Reviewer_AWXpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ScaleIMTS, a framework for adapting pretrained foundation models to irregular multivariate time series with locally varying sampling scales. The method combines segment-level scale-aware expert routing, an event-patch encoder for asynchronous observations and cross-variable interactions, and query-conditioned affine refinement of pretrained representations.

Critical strengths
  1. The paper identifies an interesting and practically relevant challenge in irregular multivariate time series modeling: local sampling scales can vary across temporal regions and variables within the same sequence. The proposed framework provides a coherent approach to adapting pretrained temporal representations to such heterogeneity by combining scale-aware expert routing with explicit modeling of asynchronous observations. The overall motivation is clear, and the methodological components are reasonably connected to the identified problem.

  2. The experimental evaluation is comprehensive, covering four forecasting and three classification datasets with comparisons against numerous recent methods. ScaleIMTS achieves strong performance across both tasks, demonstrating the potential of adapting pretrained foundation representations to irregular time series. Furthermore, the ablation studies and local-scale perturbation experiments provide supporting evidence for the effectiveness and robustness of the proposed framework.

Critical weaknesses
  1. The independent contribution of the pretrained foundation prior is not adequately isolated. In Table 3, the "w/o Foundation Prior" variant removes both the pretrained temporal encoder and the Scale Expert Router, making it difficult to attribute the substantial performance degradation specifically to pretrained temporal knowledge. Furthermore, the pretrained ISTS-PLM baseline is included in forecasting experiments but omitted from classification comparisons, leaving the benefits of foundation-prior adaptation less thoroughly established across downstream tasks.

  2. The scale-specific specialization of the three experts remains insufficiently validated. Although the experts are designated as short-, mid-, and long-term, they are implemented as independent MLPs operating on different summary statistics derived from the same temporal representations. The current ablations demonstrate the usefulness of segment-level routing, but do not establish whether the experts learn distinct temporal-scale preferences or whether the improvements primarily arise from additional model capacity and flexible feature aggregation.

  3. Table 2 omits ISTS-PLM from the classification experiments, although the paper follows its associated evaluation protocol and presents foundation-prior adaptation as a central contribution. Moreover, Appendix F.1 states that ScaleIMTS uses Timer, whereas Appendix D describes ISTS-PLM as using a GPT temporal backbone and a BERT variable backbone. Section 5.2 nevertheless describes the two methods as using “the same frozen temporal prior.” This ambiguity makes it difficult to separate improvements from the proposed adaptation mechanism from those attributable to the backbone.

Questions for authors
  1. Could the authors conduct a more controlled comparison using the same architecture with randomly initialized or task-trained temporal encoders to isolate the contribution of pretraining? Additionally, could ISTS-PLM be included in the classification experiments to provide a direct foundation-model comparison across both downstream tasks?

  2. Could the authors visualize the routing probabilities of the three experts against the actual local sampling intervals? Additionally, how does the proposed scale-aware mixture compare with a parameter-matched single-MLP adapter or a conventional mixture-of-experts module without scale-specific summary designs?

  3. Could you clarify the backbone architectures, pretrained checkpoints, and freezing settings used by both methods, and reconcile the descriptions above? Could you also include ISTS-PLM in the classification comparison and provide a forecasting comparison using the same pretrained backbone to better isolate the contribution of the proposed adaptation mechanism?

2 comments

Dear Reviewer AWXp,

Thank you for your constructive review. We have added controlled comparisons and analyses to clarify the contributions of pretrained representations, local-scale routing, and backbone choice.

Q1: Isolating the contribution of pretrained foundation knowledge

We replaced the pretrained temporal encoder with a trainable, randomly initialized standard Transformer while retaining the ScaleIMTS framework, including the Scale Expert Router, and using the same training protocol. This w/o Foundation Model control assesses the contribution of pretrained temporal representations.

Dataset (metrics)ScaleIMTSw/o Foundation Model
Human Activity (MSE / MAE)0.0425 ± 0.0003 / 0.1154 ± 0.00121.3805 ± 0.0032 / 0.9795 ± 0.0011
USHCN (MSE / MAE)0.2157 ± 0.0049 / 0.2890 ± 0.00330.6186 ± 0.1187 / 0.4980 ± 0.0283
P12 (AUROC / AUPRC, %)86.5 ± 0.6 / 52.6 ± 0.963.8 ± 0.2 / 21.5 ± 0.3
P19 (AUROC / AUPRC, %)93.4 ± 0.2 / 68.2 ± 0.385.4 ± 0.7 / 51.0 ± 0.6

ScaleIMTS is stronger on every metric across all four datasets, demonstrating the benefit of pretrained temporal representations for forecasting and classification. We also added ISTS-PLM to the classification comparison. ScaleIMTS achieves lower MSE and MAE than ISTS-PLM on all four forecasting datasets; on classification, its P12 AUROC/AUPRC gains are 10.2/12.9 points and its P19 gains are 14.0/25.3 points. It also improves all four PAM metrics by 23.8–24.4 points.

Q2: Scale-specific specialization of routing experts

We grouped held-out samples into short-, mid-, and long-interval tertiles using thresholds estimated from the training split. For each group, we disabled each expert at test time and renormalized the remaining routing weights. Entries are metric changes relative to the full model; bold marks the largest degradation per group.

DatasetInterval groupOff-ShortOff-MidOff-Long
USHCNShort+0.071+0.058+0.004
Mid+0.026+0.092+0.063
Long+0.015+0.019+0.061
Human ActivityShort+0.0072+0.0010+0.0006
Mid+0.0011+0.0058+0.0034
Long+0.0007+0.0013+0.0047
P12Short-0.186-0.028-0.014
Mid-0.021-0.079-0.025
Long-0.012-0.026-0.099
P19Short-2.119-1.181-0.693
Mid-1.225-2.853-0.831
Long-0.940-2.312-3.114

Across all four datasets and interval groups, removing the expert matched to the group produces the largest performance degradation, demonstrating distinct expert contributions across sampling regimes. The analysis is reported in Fig.9 in our paper.

Q3: Foundation-backbone comparison

“The same frozen temporal prior” was a typo and has been removed. The revised manuscript clarifies that ScaleIMTS uses a frozen Timer encoder, while ISTS-PLM uses a GPT temporal backbone and a BERT variable backbone. We added ScaleIMTS-P, which uses the ISTS-PLM backbone within the ScaleIMTS framework, to compare backbone choices while retaining the proposed adaptation.

MethodActivity MSEActivity MAEUSHCN MSEUSHCN MAEPhysioNet MSEPhysioNet MAEMIMIC-III MSEMIMIC-III MAE
ISTS-PLM0.0746 ± 0.00160.1779 ± 0.00480.3266 ± 0.06950.3492 ± 0.03910.4408 ± 0.16340.4644 ± 0.10280.5207 ± 0.01780.4513 ± 0.0106
ScaleIMTS-P0.0659 ± 0.00260.1552 ± 0.00370.3013 ± 0.07960.3121 ± 0.00680.4237 ± 0.09240.4599 ± 0.10100.5378 ± 0.01190.4967 ± 0.0078
ScaleIMTS0.0425 ± 0.00030.1154 ± 0.00120.2157 ± 0.00490.2890 ± 0.00330.2988 ± 0.00140.3656 ± 0.00190.4255 ± 0.00810.3871 ± 0.0025

ScaleIMTS outperforms ScaleIMTS-P on both metrics across all four forecasting datasets, showing that the Timer backbone is better suited to the ScaleIMTS framework and these tasks than the GPT/BERT backbone.

Thank you for your detailed response and additional experiments. I appreciate the authors' efforts to address my concerns. However, the new comparison does not fully isolate the contribution of pretraining, as the encoder architecture and freezing settings differ. Furthermore, while the expert ablations demonstrate scale-specific behavior, the lack of parameter-matched baselines still makes it difficult to establish the unique contribution of the proposed routing mechanism.

Therefore, although the rebuttal provides useful clarifications, my main concerns remain insufficiently addressed. I will maintain my original score.

Reviewer_fvM7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ScaleIMTS, a framework for irregular multivariate time series modeling that addresses local temporal scale heterogeneity. It introduces a Scale Expert Router to adapt frozen pretrained representations through short-, mid-, and long-term experts, an Event Patch Encoder to capture irregular temporal patterns and cross-variable dependencies, and a Prior Refinement Module to integrate these representations. Experiments on 7 datasets demonstrate good performance.

Critical strengths
  1. The paper addresses an interesting and practically relevant problem by considering local temporal scale heterogeneity across time and variables.

  2. The proposed framework reasonably integrates pretrained temporal representations with scale-aware adaptation and irregularity-aware modeling.

  3. The experiments are comprehensive, covering forecasting and classification tasks, component ablations, hyperparameter analyses, and robustness evaluations.

Critical weaknesses
  1. The scale-specific behavior of the experts could be better analyzed. It remains unclear whether the short-, mid-, and long-term experts specialize in different sampling scales. Analyzing routing weights across different local sampling intervals would help clarify their specialization.

  2. The reliability of the ablation results could be strengthened. While the main experiments use five random seeds, the ablations use only one. Some core components show small or inconsistent improvements on certain metrics. Reporting mean and standard deviation across multiple runs would provide more reliable evidence.

  3. The inference efficiency could be further evaluated. Although training time and computational complexity are analyzed, a direct comparison of inference latency with baselines would help assess the practical overhead of the additional modules.

Questions for authors
  1. The scale-specific behavior of the experts could be better analyzed. It remains unclear whether the short-, mid-, and long-term experts specialize in different sampling scales. Analyzing routing weights across different local sampling intervals would help clarify their specialization.

  2. The reliability of the ablation results could be strengthened. While the main experiments use five random seeds, the ablations use only one. Some core components show small or inconsistent improvements on certain metrics. Reporting mean and standard deviation across multiple runs would provide more reliable evidence.

  3. The inference efficiency could be further evaluated. Although training time and computational complexity are analyzed, a direct comparison of inference latency with baselines would help assess the practical overhead of the additional modules.

1 comment

Dear Reviewer fvM7,

Thank you for the constructive review. The revision adds expert-specialization analysis, five-seed ablations, and inference-latency comparisons to strengthen the evidence for ScaleIMTS.

Q1: Scale-specific specialization of routing experts

For each dataset, held-out samples were grouped into short-, mid-, and long-interval tertiles using thresholds from the training split. We disabled each expert separately and renormalized the remaining routing weights. Entries show changes from the full model; bold marks the largest degradation in each group.

DatasetIntervalOff-ShortOff-MidOff-Long
USHCNShort+0.071+0.058+0.004
Mid+0.026+0.092+0.063
Long+0.015+0.019+0.061
Human ActivityShort+0.0072+0.0010+0.0006
Mid+0.0011+0.0058+0.0034
Long+0.0007+0.0013+0.0047
P12Short-0.186-0.028-0.014
Mid-0.021-0.079-0.025
Long-0.012-0.026-0.099
P19Short-2.119-1.181-0.693
Mid-1.225-2.853-0.831
Long-0.940-2.312-3.114

Across all four datasets and interval groups, the largest degradation occurs when the expert matched to that interval group is disabled, showing distinct contributions across sampling regimes.

Q2: Reliability and statistical stability of ablation results

We repeated every ablation across five seeds and report mean ± standard deviation. Full ScaleIMTS leads on both forecasting metrics for Human Activity and USHCN, both P19 classification metrics, and P12 AUPRC.

VariantHuman Activity MSE / MAEUSHCN MSE / MAEP12 AUROC / AUPRC (%)P19 AUROC / AUPRC (%)
Full0.0425±0.0003 / 0.1154±0.00120.2157±0.0049 / 0.2890±0.003386.5±0.6 / 52.6±0.993.4±0.2 / 68.2±0.3
w/o Local Scale0.0496±0.0017 / 0.1351±0.00310.2963±0.0028 / 0.2994±0.002185.4±0.5 / 51.8±0.290.2±0.3 / 64.9±0.1
Global Routing0.0508±0.0048 / 0.1312±0.00260.3029±0.0006 / 0.3086±0.004286.1±0.7 / 52.4±0.491.6±0.9 / 66.5±0.5
w/o Scale Statistics0.0485±0.0039 / 0.1327±0.00510.3022±0.0022 / 0.3424±0.002886.3±1.4 / 51.1±1.790.7±0.6 / 65.8±0.9
w/o Event Patch0.0479±0.0082 / 0.1319±0.01110.2739±0.0025 / 0.3704±0.003785.1±0.7 / 51.7±1.191.6±0.1 / 64.4±0.3
w/o Foundation Prior1.3824±0.0204 / 0.9827±0.03600.7513±0.1395 / 0.5732±0.054361.9±2.3 / 21.4±1.183.3±3.9 / 49.2±3.4
w/o Foundation Model1.3805±0.0032 / 0.9795±0.00110.6186±0.1187 / 0.4980±0.028363.8±0.2 / 21.5±0.385.4±0.7 / 51.0±0.6
Additive Update0.0481±0.0009 / 0.1356±0.00060.2862±0.0019 / 0.2969±0.001086.0±0.2 / 51.2±0.390.8±0.2 / 65.4±0.5
Concat Fusion0.3127±0.0152 / 0.4192±0.01350.4925±0.0033 / 0.4082±0.005886.5±0.4 / 52.1±0.590.7±0.9 / 63.1±0.8

The five-seed results are reported in revised paper.

Q3: Inference latency and practical efficiency

We benchmarked all 16 forecasting models on four datasets, measuring single-sample forward latency and complete test-pass time at batch size one on the same hardware and protocol.

DatasetScaleIMTS forward (ms/sample)ScaleIMTS test (s)t-PatchGNN test (s)CRU test (s)
USHCN10.4341.2173.36191.530
Human Activity10.0162.2267.15375.013
PhysioNet 20129.79111.24759.075150.163
MIMIC-III21.32142.963343.966501.112

ScaleIMTS completes the test pass 2.8–8.0× faster than t-PatchGNN and 11.7–75.2× faster than CRU. The full 16-model results, including per-sample and end-to-end latency, are reported in revised paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.