acceptodds
Under review as a conference paper at ICLR 2027

QiYao-I: A Manifold Based Foundation Model for Irregular Multivariate Time Series Forecasting

Abstract

Irregular multivariate time series forecasting is a challenging yet important problem in real-world applications, where observations are often irregularly sampled and asynchronously recorded across variables. Existing time series foundation models are mostly built on regularly sampled sequences, making them difficult to generalize to irregular time intervals and asynchronous cross-variable dependencies. To address these challenges, we propose QiYao-I, a manifold based foundation model for irregular multivariate time series forecasting. Specifically, we introduce a novel sampling-conditioned temporal manifold attention mechanism that maps real timestamps into a learnable temporal manifold feature space and injects temporal manifold biases into attention layers, enabling the model to capture both irregular time intervals and local sampling structures. Further, we propose a dynamic variable interaction mechanism with frequency awareness. It selectively performs cross-variable message passing under asynchronous observations. Extensive experiments on real-world irregular multivariate forecasting benchmarks demonstrate that QiYao-I achieves superior performance compared with both time series foundation models and end-to-end irregular forecasting models, showing strong generalization ability in zero-shot and few-shot settings.

Primary area:
learning on time series and dynamical systems
Keywords:
irregular time series, manifold attention, channel aware, foundation model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_7WvaOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes QiYao-I, a foundation model for irregular multivariate time series forecasting. It introduces sampling-conditioned temporal manifold attention to capture irregular temporal dependencies and learnable spectral representations to guide asynchronous cross-variable interactions. Experiments on 8 datasets demonstrate competitive zero-shot and few-shot forecasting performance compared with existing foundation models and irregular forecasting methods.

Critical strengths
  1. The paper addresses an important and practical problem by jointly modeling irregular temporal dependencies and asynchronous cross-variable interactions.

  2. Incorporating local sampling structures into temporal attention is an interesting design that enables the model to distinguish different sampling patterns.

  3. The experiments are relatively comprehensive, covering 8 datasets with comparisons against foundation models and irregular forecasting methods, along with ablation and efficiency analyses.

Critical weaknesses
  1. The necessity of the temporal manifold formulation is not fully justified. Although the ablation studies demonstrate its effectiveness, comparisons with simpler timestamp-distance-based attention biases using the same sampling-conditioned bandwidth would better isolate the contribution of the learned manifold formulation.

  2. The spectral representations may be sensitive to irregular sampling density. In Section 3.3, the global spectral coefficients are computed by averaging point-level projections without explicitly accounting for non-uniform sampling intervals. Thus, the representations may vary with the sampling distribution even for the same underlying signal. Further analysis would help establish their robustness to different sampling patterns.

  3. The robustness to varying levels of sampling irregularity could be further evaluated. Although the paper considers different sampling patterns, it remains unclear whether QiYao-I maintains its advantages over existing methods as the degree of sampling irregularity increases or decreases. Controlled experiments under different irregularity levels would better demonstrate its robustness.

Questions for authors
  1. The necessity of the temporal manifold formulation is not fully justified. Although the ablation studies demonstrate its effectiveness, comparisons with simpler timestamp-distance-based attention biases using the same sampling-conditioned bandwidth would better isolate the contribution of the learned manifold formulation.

  2. The spectral representations may be sensitive to irregular sampling density. In Section 3.3, the global spectral coefficients are computed by averaging point-level projections without explicitly accounting for non-uniform sampling intervals. As a result, the representations may vary with the sampling distribution even for the same underlying signal. Further analysis would help establish their robustness to different sampling patterns.

  3. The robustness to varying levels of sampling irregularity could be further evaluated. Although the paper considers different sampling patterns, it remains unclear whether QiYao-I maintains its advantages over existing methods as the degree of sampling irregularity increases or decreases. Controlled experiments under different irregularity levels would better demonstrate its robustness.

6 comments
AuthorsOpenReview ↗

Response to Q1 & W1 — Learnable manifold distance under sampling-conditioned bandwidth

Thanks for your valuable suggestion. To demonstrate the contribution of the learnable temporal manifold distance, we compare it with a simpler timestamp-distance-based attention bias, while retaining the same anchor-based sampling-conditioned bandwidth. Both variants are pretrained from scratch on the same corpus with a matched number of optimizer updates, using a reduced pretraining budget due to time constraints.

  • T0: Simpler timestamp-distance-based bias. This variant directly uses the squared timestamp difference, Duw=[(t^u−t^w)/R]2D_{uw}=[(\widehat{t}_u-\widehat{t}_w)/R]^2.
  • T1:Learnable temporal manifold distance based bias. Our component presented in the Section 3.2.

Table R1.

DatasetT0: Simpler timestamp-distance-based biasT1: Learnable temporal manifold distance based bias
CESNET (10⁻³)22.41 / 90.3221.97 / 90.20
APTC (10⁻⁵)390.98 / 4634.17390.63 / 4630.02
FNSPID (10⁻⁵)356.63 / 3434.65338.43 / 3377.95
Human Activity (10⁻⁴)41.34 / 388.3341.00 / 389.23
ClusterTrace (10⁻³)41.79 / 150.5838.55 / 144.92
Seabirds (10⁻⁶)180.80 / 7574.68162.83 / 7458.76
GDELT (10⁻⁴)197.77 / 846.03203.71 / 856.29
USHCN (10⁻³)813.44 / 405.99776.70 / 404.91

T1 achieves lower MSE on seven of eight datasets. These results support the additional benefit of the learned temporal manifold formulation over a simpler timestamp-distance-based attention bias.

AuthorsOpenReview ↗

Response to Q2 & W2 — Sensitivity of spectral representations to irregular sampling density

Thanks for your valuable suggestion. We pretrain two variants from scratch on the same corpus under a common budget of same optimizer updates. Due to time constraints, we adopt a limited pretraining budget for these ablation experiments. Our downstream datasets have different sampling patterns, which is detailed in Table 7. We compare time-weighted aggregation with our equal-weight aggregation. As shown in Table R1, equal-weight aggregation achieves lower errors on 10 out of 16 dataset. These results indicate that equal-weight aggregation already provides an effective representation of spectral information under irregular sampling, and explicitly incorporating time-weighted aggregation does not necessarily improve forecasting performance.

Table R1.

DatasetEqual-weight aggregationTime-weighted aggregation
CESNET (10⁻³)22.17 / 90.3722.19 / 90.52
APTC (10⁻⁵)392.50 / 4640.84392.38 / 4639.72
ClusterTrace (10⁻³)38.55 / 144.9238.86 / 144.86
Seabirds (10⁻⁶)162.83 / 7458.76169.70 / 7618.67
Human Activity (10⁻⁴)41.20 / 391.1840.97 / 390.14
FNSPID (10⁻⁵)324.98 / 3331.04326.56 / 3334.38
GDELT (10⁻⁴)201.29 / 856.52202.36 / 854.18
USHCN (10⁻³)768.79 / 404.17774.37 / 406.61
AuthorsOpenReview ↗

Response to Q3 & W3 — Robustness to varying degrees of sampling irregularity 1

Thanks for your valuable suggestion. Following your suggestions, we add two experiments.

Reducing the number of historical observations. For each input window, we varied the retention rate from 100% to 10%, retaining the corresponding percentage of historical observations while preserving the first and last observations. We compare QiYao-I with Chronos-2 on identical retained observations and future targets. Chronos-2 uses the original linear-interpolation adaptation.

Table R1.

History retainedModelAPTC (10⁻⁵)ClusterTrace (10⁻³)Seabirds (10⁻⁶)FNSPID (10⁻⁵)CESNET (10⁻³)
100%QiYao-I385.30 / 4597.0937.22 / 142.67165.67 / 7411.77249.80 / 3016.1222.01 / 90.24
Chronos-2401.85 / 4681.7545.99 / 157.43295.50 / 9384.00311.00 / 3253.6022.00 / 90.94
90%QiYao-I386.39 / 4603.1837.25 / 142.81167.51 / 7447.23250.62 / 3021.4122.04 / 90.24
Chronos-2399.31 / 4666.2145.46 / 156.56292.31 / 9358.46306.44 / 3233.9621.99 / 90.81
80%QiYao-I387.63 / 4610.0637.27 / 142.88170.08 / 7478.49253.88 / 3032.9822.03 / 90.19
Chronos-2399.09 / 4664.7744.76 / 155.83302.36 / 9570.26302.94 / 3227.6222.00 / 90.82
70%QiYao-I389.10 / 4618.6937.23 / 142.89170.65 / 7471.20256.95 / 3052.1722.01 / 90.27
Chronos-2397.82 / 4658.4244.75 / 155.60325.17 / 9667.50302.31 / 3240.9222.05 / 90.93
60%QiYao-I391.28 / 4631.5037.59 / 143.31169.80 / 7433.18260.72 / 3071.5522.15 / 90.39
Chronos-2399.44 / 4666.9944.57 / 155.36319.17 / 9058.79298.28 / 3227.2522.10 / 90.99
50%QiYao-I394.39 / 4649.1237.71 / 143.73168.06 / 7388.24266.03 / 3100.9322.28 / 90.60
Chronos-2402.28 / 4682.2145.00 / 155.94300.70 / 9130.18297.12 / 3245.1322.13 / 91.17
40%QiYao-I398.41 / 4672.0238.21 / 144.92166.40 / 7390.86271.61 / 3118.0322.50 / 90.87
Chronos-2406.75 / 4707.3144.13 / 154.89337.75 / 9944.06287.61 / 3238.4722.25 / 91.41
30%QiYao-I404.88 / 4708.1537.96 / 144.40170.65 / 7534.77277.56 / 3148.2022.82 / 91.20
Chronos-2412.92 / 4737.8243.25 / 153.25206.21 / 7504.69320.62 / 3372.1122.55 / 91.80
20%QiYao-I415.29 / 4766.4938.27 / 145.00164.76 / 7400.93286.48 / 3181.0022.99 / 91.66
Chronos-2425.98 / 4812.5243.03 / 153.23214.68 / 8351.13302.13 / 3322.0223.00 / 92.86
10%QiYao-I438.26 / 4892.1740.83 / 147.93162.10 / 7321.07310.15 / 3287.5623.37 / 92.48
Chronos-2459.45 / 5007.4644.10 / 154.80250.22 / 9705.68369.79 / 3904.6324.31 / 94.72

Progressively removing historical observations while preserving the endpoints creates sparser inputs with more gaps. QiYao-I remains robust to moderate thinning, with MSE increasing by at most 2.66% at 80% retention and 6.50% at 50% retention across the five datasets. Compared with Chronos-2, QiYao-I achieves lower MSE on four of the five datasets at both retention levels, with average relative reductions of 15.88% and 14.41%, respectively.

Reviewer_jT35OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes QiYao-I, a foundation model for irregular multivariate time-series forecasting. It combines sampling-conditioned temporal manifold attention with learned spectral features and frequency-guided cross-variable patch interaction. The experiments cover zero-shot and 10%-few-shot forecasting on eight datasets, with comparisons against regular-input foundation models, an irregular foundation model, and end-to-end irregular forecasters. The reported results are promising, but the available evidence is not yet sufficient to support the paper's strongest claims about generalization and superiority.

Critical strengths
  • The paper addresses an important and relatively underexplored problem: forecasting with irregular timestamps and asynchronous variables. Directly processing irregular observations may be valuable in applications where interpolation is undesirable.
  • The architecture is technically ambitious and includes useful diagnostic studies, including comparisons involving bandwidth, routing, and pretrained versus randomly initialized models.
Critical weaknesses

Major Issues

  1. The pretraining provenance and contamination issue is not sufficiently established. The claim that there is “no test data leakage” is not supported by a dataset-level overlap audit. The paper pretrains on GIFT-Eval, Chronos-related corpora, and a collected irregular corpus, but does not specify the exact source contents, versions, deduplication rules, checks for transformed records, or excluded time ranges. This is decision-critical for the zero-shot claim; the concern is the absence of evidence, not proof that leakage actually occurred.
  2. The main comparisons contain confounding factors. In Table 2, QiYao-I uses 10% of the downstream data, whereas the end-to-end irregular baselines use 100%. The regular-input foundation models receive interpolated inputs, while QiYao-I receives native irregular inputs. These protocols may demonstrate practical data efficiency, but they do not establish model superiority under controlled conditions.

Minor Issues

  • Table 1 and several appendix tables are difficult to audit because of alignment and labeling problems; “ClusterTrace (?)” remains unresolved.
  • The definitions of the irregularity statistics and dataset construction are insufficient, including whether the irregularity is native or synthetically induced.
  • The names, versions, preprocessing procedures, and checkpoint sources of the baselines are inconsistent or incomplete.
  • The paper trains quantiles but evaluates only median point forecasts; this is secondary to the issues above.
  • Claims such as “superior performance” and “strong generalization” should be narrowed to the protocols that were actually validated.
  • The mathematical interpretation of the spectral component remains insufficiently justified: in Eqs. (10)–(11), value-weighted sinusoidal features at historical timestamps are aggregated and then rotated using future timestamps; the manuscript does not sufficiently explain why this constitutes valid spectral extrapolation rather than merely a learned timestamp-dependent feature.
  • Scale and reproducibility reporting is incomplete: the paper omits the parameter count, pretraining windows/tokens, optimization steps, batch size, checkpoint-selection procedure, source-wise contributions, and complete downstream tuning details; the main results also lack uncertainty estimates.
  • The ablations do not sufficiently isolate the major architectural choices, and the routing gains in Tables 3, 9, and 11 are small and inconsistent.
  • Information flow and normalization are ambiguous: Section 3.4 does not clearly specify future-patch interactions, and the relationship between the instance normalization in Eq. (1) and the dataset-level normalization in Appendix A.2 is unclear.
  • QiYao-I appears to be a meaningful and interesting Chinese method name with philosophical connotations. However, because the name is presented in pinyin for this venue, the meaning and implications originally carried by the Chinese characters are lost; even Chinese readers seeing the method name would not know whether it refers to “七曜” (the Seven 'Yao', seems time series related, but what is "I" ), “姚期智” (Yao Qi zhI, the Turing Award laureate), “爱祁瑶” (a possible personal name), or “齐腰” (waist-high / reaching the waist). I suggest that the authors add a Cultural Implication section in the appendix, briefly explaining the original Chinese characters corresponding to the pinyin and the source of any relevant allusion. This would help readers understand the connotations behind the method name, make the method more memorable, and improve the overall quality of the paper.

Reviewer Limitations This assessment is based on the supplied manuscript and reviews. It seems there isn't any code supplement.

Questions for authors

Questions For Authors:

  • Clarification that could change the assessment: What are the exact identities and versions of all pretraining sources? For each downstream dataset, what record-level, entity-level, timestamp-level, and transformed-data overlap checks were performed? How many records or windows were removed?
  • What is the precise derivation and intended meaning of Eqs. (10)–(11)? Do the results remain valid against irregular least-squares Fourier estimation, timestamp-only features, and a version without future spectral features?
  • What are the exact source-target masks for historical and future patches in Section 3.4? Could any future-to-future or target-derived information enter message passing, patch construction, or normalization?
  • Can the main comparisons be rerun with matched downstream budgets (all methods using 10%, and all methods using 100%), with the interpolation grid and preprocessing clearly specified?
  • What are the parameter count, number of pretraining windows/tokens, optimization steps, batch size, checkpoint-selection rule, and uncertainty estimates for the headline results?
  • Could the remaining issues concerning scale, ablations, information flow, normalization, and table presentation be clarified in the revision? These are primarily fixable reporting and validation improvements, but they would help assess whether the two central issues above materially change the paper’s conclusions.
12 comments
AuthorsOpenReview ↗

Response to Q1 & Main W1— Pretraining data provenance and overlap

Thanks for your valuable suggestion. We add Appendix A.6 in the revised manuscript, and list the detailed pretraining data and distinguishes the original and filtered manifests. The five source collections are GIFT-Eval Pretrain, GIFT-Eval training data, the Chronos training corpus, and our corpus. The archived manifests show the local dataset configurations, which are used in training. There is no test-data leakage.

AuthorsOpenReview ↗

Response to Q4 & Main W2. Fairness of the experimental comparison

Thanks for your valuable suggestion. The original Table 2 compared QiYao-I fine-tuned on 10% of the downstream training windows with irregular forecasting baselines trained on 100%, with the purpose that even with 10% training data, QiYao-I could outperform these baselines trained with 100% data. We supplemented comparisons where APN, Hi-Patch, and tPatchGNN are trained from scratch on 10% of the training data and fine-tuned QiYao-I on 100%.

Table R1. Results with 10% of the downstream training windows.

DatasetQiYao-IAPNHi-PatchtPatchGNN
CESNET (10−310^{-3})21.59 / 87.4951.10 / 149.3623.55 / 102.6523.17 / 102.86
APTC (10−510^{-5})370.92 / 4507.52592.54 / 5904.66375.71 / 4540.35398.68 / 4727.97
ClusterTrace (10−310^{-3})36.57 / 142.8149.65 / 165.6146.37 / 164.7948.98 / 161.84
Seabirds (10−610^{-6})171.39 / 7810.57132892.28 / 350185.752505.83 / 40484.644073.20 / 54376.09
Human Activity (10−410^{-4})39.52 / 368.79190.54 / 1096.53115.05 / 826.1340.82 / 404.17
FNSPID (10−510^{-5})476.13 / 3916.123833.18 / 17322.692195.52 / 10918.282571.27 / 11017.74
GDELT (10−410^{-4})188.08 / 848.31977.24 / 2747.94207.86 / 921.72202.34 / 930.44
USHCN (10−310^{-3})538.82 / 253.34594.98 / 368.60538.11 / 338.60631.70 / 383.57

Table R2. Results with 100% of the downstream training windows.

Dataset (scale)QiYao-IAPNHi-PatchtPatchGNN
CESNET (10−310^{-3})21.01 / 86.5423.30 / 101.3524.80 / 104.0523.45 / 103.10
APTC (10−510^{-5})366.52 / 4472.86385.00 / 4640.00385.00 / 4630.00590.00 / 5910.00
ClusterTrace (10−310^{-3})36.59 / 144.6443.60 / 153.6043.25 / 154.8045.40 / 155.70
Seabirds (10−610^{-6})174.96 / 7632.591050.00 / 26100.00108900.00 / 281500.001200.00 / 24350.00
Human Activity (10−410^{-4})38.00 / 363.6837.00 / 375.0039.00 / 411.5037.50 / 387.50
FNSPID (10−510^{-5})485.29 / 3923.791375.00 / 8145.002295.00 / 10405.00915.00 / 7420.00
GDELT (10−410^{-4})191.60 / 825.13190.00 / 855.50192.50 / 879.50191.50 / 835.50
USHCN (10−310^{-3})524.11 / 246.04518.60 / 333.35556.75 / 386.70665.60 / 437.40

Among the four methods shown, QiYao-I obtains the lowest error on 15 of 16 metrics at 10% and 13 of 16 at 100%.

Downstream splits and preprocessing. We use 60/20/20 train, validation, and test splits. The baseline runs use at most 100 epochs with validation-based early stopping and patience 3. The details are shown in Appendix A.2.

Interpolation grid for regular-input foundation models. The irregular models directly receive irregular observations and masks. For zero-shot comparisons with regular-input foundation models, a shared adapter maps each irregular historical window onto an evenly spaced temporal grid using linear interpolation. Forecasts are then mapped back to the original query timestamps and evaluated only at valid targets. Further details are provided in Appendix A.3.

Input adaptation for regular-input baselines. In added Appendix C.7, we compare different input adaptation methods for regular-input baseline. The results suggest that simple linear interpolation is already an effective adaptation strategy for regular-input foundation models, and more complex preprocessing does not necessarily lead to better forecasting accuracy.

AuthorsOpenReview ↗

Response to Q2 and Minor W6 — Derivation and design of Eqs. (10) and (11)

Thanks for your valuable suggestion. We clarify why the mechanism constitutes spectral extrapolation, provide the precise derivation and intended meaning of Eqs. (10)–(11), and compare the mechanism with irregular least-squares Fourier estimation, timestamp-only features, and a version without future spectral features.

Why this constitutes spectral extrapolation. We use the term spectral extrapolation to describe extending a history-conditioned spectral representation to future timestamps. Specifically, historical values and timestamps jointly determine the spectral coefficients, while future timestamps advance their phases without changing their magnitudes. Unlike timestamp-only features, the resulting future representations depend on historical values, where histories with identical timestamps but different values generally yield different spectral features. This provides a history-conditioned extrapolation in the spectral space rather than merely a learned timestamp-dependent feature. We do not claim exact Fourier spectrum recovery under arbitrary irregular sampling. Instead, the component constructs learned future spectral features that are fused with temporal representations for forecasting.

Derivation of Eqs. (10)–(11). For frequency wkw_k, Eq. (10) multiplies each historical value by a sinusoidal basis at its timestamp:

Cv,i,kh=mv,ihzv,iexp⁡(−i2πwkt^v,ih).C^h_{v,i,k} = m^h_{v,i} z_{v,i} \exp\left(-\mathrm{i}2\pi w_k \widehat{t}^h_{v,i}\right).

Euler’s formula gives the cosine and negative sine terms. Averaging over the nvn_v valid observations produces Cˉv,k\bar{C}_{v,k}, which summarizes how historical values align with that frequency. Equation (11) multiplies Cˉv,k\bar{C}_{v,k} by exp⁡(i2πwkt^v,jp)\exp\left(\mathrm{i}2\pi w_k \widehat{t}^p_{v,j}\right), advancing its phase to the future query time. Expressing this multiplication in real and imaginary parts gives the rotation matrix. Combining both equations, for nv>0n_v > 0,

Cv,j,kp=1nv∑i:mv,ih=1zv,iexp⁡[i2πwk(t^v,jp−t^v,ih)].C^p_{v,j,k} = \frac{1}{n_v} \sum_{i:m^h_{v,i}=1} z_{v,i} \exp\left[\mathrm{i}2\pi w_k \left(\widehat{t}^p_{v,j}-\widehat{t}^h_{v,i}\right)\right].

Thus, the future spectral feature depends on historical values and their time differences to the query. Comparison with the three variants. We pretrain the variants from scratch on the same corpus with a matched number of optimizer updates, using a reduced pretraining budget due to time constraints. The variants differ only in how the future spectral representations are computed before being concatenated with the time-domain representations.

  • Irregular LS Fourier: jointly estimates sine and cosine coefficients using ridge least squares on valid historical observations.
  • Timestamp only: uses sinusoidal features of the future query timestamp without historical value conditioning.
  • No future spectral features: sets the spectral input to the forecasting head to zero.

Table R1. Spectral representation controls.

Dataset (scale)Irregular LS FourierTimestamp onlyNo future spectral featuresOur model
CESNET (10−310^{-3})22.24 / 90.7022.67 / 91.4222.21 / 90.9522.17 / 90.37
Seabirds (10−610^{-6})174.05 / 7553.60186.97 / 7959.42173.85 / 7495.65162.83 / 7458.76
FNSPID (10−510^{-5})327.14 / 3359.41398.48 / 3611.45343.01 / 3407.45324.98 / 3331.04
APTC (10−510^{-5})392.89 / 4639.91392.59 / 4641.91392.23 / 4637.12391.50 / 4635.24

The full model achieves lower MSE and MAE than all three variants, showing the effects of our compoent. Thus, our results in the paper remain valid.

Reviewer_WHqxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This work proposed a foundation model for irregular multivariate time series forecasting. The proposed method combines sampling-conditioned temporal attention, spectral representations computed directly from irregular observations, and selective interaction between patches from different variables. I find the overall design reasonable and the empirical evaluation substantial. The contribution primarily lies in integrating these mechanisms for transferable irregular forecasting. My reservations concern the attribution of improvements to individual components, the interpretation of the ablation results, and methodological clarity.

Critical strengths

The temporal attention mechanism accounts for both timestamps and local observation patterns, while the interaction module addresses dependencies between asynchronously observed variables. Using temporal proximity to identify candidate patches and frequency similarity to weight their messages is a sensible design.

The experiments cover eight datasets and include comparisons with both forecasting foundation models and models specifically designed for irregular time series. The improvements on datasets such as Seabirds and ClusterTrace are substantial. The results using only 10% of the downstream training data also provide useful evidence of adaptation with limited target data.

The bandwidth interventions, comparisons with identical temporal neighbors, and pre-trained versus random initialization experiments help explain how the model behaves. In particular, the experiments that preserve endpoint timestamps and observation counts while changing the sampling pattern support the claim that the mechanism responds to local sampling structure.

Critical weaknesses
  1. The notation in the paper is overly complicated and sometimes overloaded, making the proposed method difficult to follow. For example, rr denotes both an attention head index and the sampling-conditioned scale in ruℓ,rr_u^{\ell,r}, while RR represents both the time-normalization reference scale and the rotary transformation R(t^)R(\hat t). Wx\textbf{W}_x represents transformation weight in eq. (1) and WW represents the width of bandwidth, and W\textbf{W} with other index reprents transformation weight again. The numerous indices for layers, heads, variables, and temporal positions further obscure the relationship between the sampling descriptor, adaptive bandwidth, and attention bias in Eqs. (2)–(9). I recommend using distinct symbols for different concepts, providing a concise notation table, and first presenting the mechanism for a single variable and a single attention head before introducing the full formulation.

Although I am trying to read the paper without the help of AI and understand the authors wish to demonstrate their method precisely, but the presentation quality of notation forces me to use AI to simplify the notation in order to read it.

  1. I would like to see a separate ablation of Section 3.3. The spectral branch provides both prediction features and routing information, but their contributions are not disentangled. Removing each use separately, and then both, under comparable training settings would clarify where the gains come from. Table 3 already covers several temporal and interaction choices, so this seems to be the main missing component ablation.

  2. The evidence for the proposed temporal distance and sampling descriptor could be stronger. Table 9 supports adaptive bandwidth, but does not establish whether learned timestamp distances outperform actual time differences, or whether the anchor-based descriptor improves on observation counts and elapsed-time features. Sections B.4 and B.5 also modify a fixed checkpoint; simpler alternatives might perform differently if trained with them from the beginning. Were the Table 3 variants trained separately?

  3. The cross-variable selection and weighting deserve a more controlled comparison. Section B.2 varies K but leaves the temporal selection rule unchanged, so it does not establish its advantage over other neighbor choices. Meanwhile, Table 11 reports only about 0.39% and 1.09% MSE improvement over uniform weighting, and local-plus-global routing is not consistently best. Comparing temporal and content-based selection at matched K, and retraining content-based weighting with fixed neighbors, would help isolate these choices. Variability estimates are particularly important for the smaller gains.

Questions for authors

How much does Section 3.3 contribute through its prediction features versus its routing features? Have you tested these two uses separately, as well as removing both?

What do the learned temporal distance and anchor-based sampling descriptor add over simpler alternatives, such as actual time differences, observation counts, and elapsed time since the last observation?

Can you separate the benefit of temporal neighbor selection from frequency weighting? Comparisons with content-based selection at the same K, and content-based weighting with identical neighbors, would help. Given Table 11, is combining local and global frequency features consistently beneficial across repeated runs?

6 comments
AuthorsOpenReview ↗

Response to Q1 & W2— Contributions of the spectral branch

Thanks for your insightful suggestion. We pretrain four configurations from scratch on the same corpus under a common budget of same optimizer updates. Due to time constraints, we adopt a limited pretraining budget for these ablation experiments. Forecasting only supplies extrapolated spectral features to the prediction head and assigns uniform weights to the selected patches. Routing only uses local and global frequency similarity to weight the selected patches and sets the spectral input to the prediction head to zero. Neither disables both uses, and Both retains both. All four configurations use the same temporal neighbor selection procedure.

Table R1. Spectral branch ablation.

Dataset (scale)NeitherForecasting onlyRouting onlyBoth
CESNET (10−310^{-3})22.22 / 90.9322.15 / 90.3622.21 / 90.9522.13 / 90.31
ClusterTrace (10−310^{-3})39.36 / 146.3038.45 / 144.7639.20 / 145.9637.95 / 144.68
Seabirds (10−610^{-6})174.31 / 7507.26161.76 / 7424.28173.85 / 7495.65161.75 / 7407.99
FNSPID (10−510^{-5})351.34 / 3432.47330.46 / 3349.94343.01 / 3407.45324.98 / 3331.04

The ablation shows that spectral features are useful in two ways. Extrapolated spectral features provide periodic information directly to the prediction head and yield the larger standalone gains overall. Frequency similarity also helps the model decide how much information to aggregate from each temporally selected patch. The matched-step results show that these two uses can complement each other.

AuthorsOpenReview ↗

Response to Q2 & W3 — Learnable temporal distance and anchor-based sampling descriptor

Thanks for your insightful suggestion. We pretrain five variants from scratch on the same corpus under a common budget of same optimizer updates. Due to time constraints, we adopt a limited pretraining budget for these ablation experiments. We compare learnable temporal manifold distances with actual time differences, and compare the anchor-based sampling descriptor with observation counts and elapsed-time features.

Learnable Temporal Manifold Distance. An actual time difference treats two pairs of observations at different positions identically when their intervals are equal. The learnable temporal manifold distance treats equal intervals at different positions differently. In the comparisons in Table R2, the learnable temporal manifold distance lowers MSE in both two settings: T0 vs.T1 and T3 vs. T4, which demonstrate that the learnable temporal manifold distances outperform actual time differences.

Anchor-based sampling descriptor. The anchor responses distinguish different local sampling structures by using the Gaussian kernels even when observation counts and elapsed time are identical. In Table R2, we compare the count-based descriptor with anchor-based descriptor (T2 vs. T4). Anchor-based descriptor lowers both MSE and MAE, indicating that the anchor-based descriptor can improve forecasting performance.

The Table 3 variants in the paper are trained from srcatch separately, which have different settings compared with the rebuttal.

Table R2. Distance and sampling descriptor controls.

ConfigurationDetailsCESNET ×10−3\times 10^{-3}APTC ×10−5\times 10^{-5}FNSPID ×10−5\times 10^{-5}Human Activity ×10−4\times 10^{-4}
T0Actual time differences+fixed bandwidth22.56 / 90.48391.64 / 4638.29358.70 / 3441.1441.50 / 389.62
T1Learnable temporal manifold distance + fixed bandwidth22.20 / 90.41391.32 / 4635.92351.38 / 3432.6641.10 / 389.57
T2Learnable temporal manifold distance + count-based descriptor with elapsed-time features22.19 / 90.39391.31 / 4635.81350.89 / 3430.3741.04 / 389.25
T3Actual time differences + anchor-based descriptor22.41 / 90.32390.98 / 4634.17356.63 / 3434.6541.34 / 388.33
T4Learnable temporal manifold distance + anchor-based descriptor21.97 / 90.20390.63 / 4630.02338.43 / 3377.9541.00 / 389.23
AuthorsOpenReview ↗

Response to Q3 & W4— Separating Temporal Neighbor Selection and Frequency Weighting

Thanks for your insightful suggestion. We pretrain several variants from scratch on the same corpus under a common budget of same optimizer updates. Due to time constraints, we adopt a limited pretraining budget for these ablation experiments. We conduct additional experiments that separately examine neighbor selection and frequency-based message weighting.

1. Temporal vs. content-based neighbor selection.

We first compare two strategies for selecting neighboring patches. Temporal selection ranks candidate patches according to the temporal distance between their corresponding time intervals. Content-based selection ranks them by cosine similarity between their pooled temporal representations.

DatasetTemporal selectionContent-based selection
CESNET (10−310^{-3})21.97 / 90.2022.09 / 90.27
APTC (10−510^{-5})390.63 / 4630.02392.47 / 4641.75
ClusterTrace (10−310^{-3})38.55 / 144.9238.77 / 145.49
FNSPID (10−510^{-5})338.43 / 3377.95341.28 / 3427.30

Temporal selection achieves lower MSE/MAE on all four datasets. These results suggest that temporal proximity provides a useful neighbor selection prior.

2. Frequency weighting with identical temporal neighbors.

We next fix the neighbor sets selected by temporal distance and compare five weighting strategies: Uniform, which assigns equal weights to all selected neighbors; Content-based, which derives weights from patch representation similarity; Local only, which uses local frequency similarity; Global only, which uses global frequency similarity; and Local + Global, which combines both frequency signals.

The table reports the mean ±\pm standard deviation across three seeds for MSE / MAE.

WeightingCESNET (10−310^{-3})APTC (10−510^{-5})Seabirds (10−610^{-6})FNSPID (10−510^{-5})
Uniform22.31±0.2322.31 \pm 0.23 / 90.47±0.3590.47 \pm 0.35391.81±2.95391.81 \pm 2.95 / 4635.79±20.354635.79 \pm 20.35160.90±8.34160.90 \pm 8.34 / 7238.94±198.977238.94 \pm 198.97336.69±13.94336.69 \pm 13.94 / 3364.94±71.143364.94 \pm 71.14
Content-based22.35±0.1222.35 \pm 0.12 / 90.53±0.2690.53 \pm 0.26392.30±2.87392.30 \pm 2.87 / 4639.79±20.484639.79 \pm 20.48166.53±5.88166.53 \pm 5.88 / 7387.07±163.677387.07 \pm 163.67337.73±14.06337.73 \pm 14.06 / 3364.61±70.463364.61 \pm 70.46
Local only22.32±0.1922.32 \pm 0.19 / 90.47±0.3190.47 \pm 0.31391.54±2.46391.54 \pm 2.46 / 4634.67±17.414634.67 \pm 17.41161.56±8.35161.56 \pm 8.35 / 7262.56±219.257262.56 \pm 219.25335.24±13.84335.24 \pm 13.84 / 3359.75±64.453359.75 \pm 64.45
Global only22.33±0.2022.33 \pm 0.20 / 90.48±0.3290.48 \pm 0.32391.66±2.78391.66 \pm 2.78 / 4635.05±19.304635.05 \pm 19.30161.63±7.90161.63 \pm 7.90 / 7263.72±202.907263.72 \pm 202.90334.76±14.51334.76 \pm 14.51 / 3357.97±68.553357.97 \pm 68.55
Local + Global22.30±0.2322.30 \pm 0.23 / 90.45±0.3390.45 \pm 0.33391.53±2.58391.53 \pm 2.58 / 4634.51±18.134634.51 \pm 18.13160.88±8.13160.88 \pm 8.13 / 7225.48±166.867225.48 \pm 166.86334.68±14.34334.68 \pm 14.34 / 3357.49±67.753357.49 \pm 67.75

Local + Global achieves the lowest three-seed mean errors on all four datasets. These results show that combining local and global frequency information offers average improvements over other weighting strategies.

Reviewer_hpYiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies forecasting for irregular multivariate time series, where observations are sampled at uneven times and different variables may not be aligned. The proposed model, QiYao-I, is positioned as a foundation model for this setting. It introduces a sampling-conditioned temporal manifold attention module that maps timestamps into a learnable temporal space and uses manifold-distance-based attention bias to represent irregular temporal relations. It also proposes a frequency-guided cross-variable interaction mechanism that selects temporally nearby patches from other variables and weights them using local and global frequency similarity.

Critical strengths

The paper addresses a timely and useful problem. Most time series foundation models still assume regular sampling or rely on preprocessing to handle irregular observations. The paper makes a reasonable attempt to design the architecture around irregular timestamps and asynchronous variables instead of treating irregularity as only a data preprocessing issue.

The experimental scope is also a positive aspect. The paper evaluates on multiple datasets from different domains and compares against both foundation models and specialized irregular time series models. The zero-shot and few-shot results are particularly relevant for the foundation model claim, and the ablation study gives some evidence that both the temporal manifold bias and the frequency-guided variable interaction contribute to performance.

Critical weaknesses

The main concern is that some parts of the method are quite complex, and it is not always clear which design choices are essential. The temporal manifold attention, sampling descriptor, learnable spectral projection, patch selection, and gated message injection all add components with several hyperparameters. The ablation study is helpful, but it does not fully isolate some choices, such as the form of the sampling descriptor, the learned manifold distance, the spectral extrapolation design, and the top-k patch selection strategy. This makes it somewhat hard to judge whether the gains come from the proposed modeling principles or from accumulated architectural capacity and tuning.

The comparison with existing foundation models also needs more care. Several baselines are originally designed for regular time series, so their performance depends strongly on how irregular inputs are converted or interpolated. The paper mentions interpolation choices, but the main results would be more convincing if the preprocessing and adaptation of these baselines were described more transparently and tested with stronger alternatives. Since the claimed contribution is a foundation model for irregular time series, the fairness of this comparison is important for assessing the significance of the results.

Questions for authors

Could the authors provide a more detailed ablation on the sampling-conditioned temporal manifold attention? In particular, it would be useful to separate the effect of continuous-time RoPE, learned manifold distance, and the sampling-conditioned bandwidth descriptor.

How sensitive is the model to the number of selected cross-variable patches and to the frequency similarity weighting? I would like to know whether the method remains stable when the number of variables, sampling density, or degree of asynchrony changes.

For regular time series foundation model baselines, could the authors clarify exactly how irregular inputs are transformed before inference, and whether multiple interpolation or masking strategies were tried? This would help assess whether the improvement mainly comes from better irregular modeling or from baseline adaptation choices.

Could the authors provide more information about overlap between pretraining sources and downstream datasets, including any filtering procedure? The paper says there is no test leakage, but more details would make the zero-shot results easier to trust.

6 comments
AuthorsOpenReview ↗

Response to Q1 and W1 — Ablation of Sampling-Conditioned Temporal Manifold Attention

Thanks for your insightful suggestions. To analyze the contributions of continuous-time positional encoding, learnable temporal manifold distance, and sampling-conditioned bandwidth, we pretrain T0–T5 from scratch on the same corpus under a common budget of same optimizer updates. Due to time constraints, we adopt a limited pretraining budget for these ablation experiments. We report MSE / MAE averaged equally over the two forecast horizons.

Table R1. Ablation of Sampling-Conditioned Temporal Manifold Attention.

ConfigurationChange relative to previous configurationCESNET ×10−3\times 10^{-3}APTC ×10−5\times 10^{-5}FNSPID ×10−5\times 10^{-5}Human Activity ×10−4\times 10^{-4}
T0Standard RoPE22.58 / 91.28401.68 / 4688.88361.16 / 3523.6753.44 / 478.42
T1Continuous-time RoPE22.57 / 91.11392.67 / 4638.70358.94 / 3515.6643.54 / 408.04
T2Continuous-time RoPE + squared-time distance + fixed bandwidth22.56 / 90.48391.64 / 4638.29358.70 / 3441.1441.50 / 389.62
T3Continuous-time RoPE + learnable temporal manifold distance + fixed bandwidth22.20 / 90.41391.32 / 4635.92351.38 / 3432.6641.10 / 389.57
T4Continuous-time RoPE + learnable temporal manifold distance + count-based bandwidth22.19 / 90.39391.31 / 4635.81350.89 / 3430.3741.04 / 389.25
T5 (Full model)Continuous-time RoPE + learnable temporal manifold distance + sampling-conditioned bandwidth21.97 / 90.20390.63 / 4630.02338.43 / 3377.9541.00 / 389.23
  1. Continuous-time RoPE. To examine the contribution of continuous-time RoPE, we compare it (T1) with Standard RoPE (T0). As shown in Table R1, T1 reduces MSE by 5.36% on average, showing that positioning actual timestamps rather than observation order benefits forecasting by allowing attention to distinguish observations with different intervals.
  2. Learnable Temporal Manifold Distance. To examine whether learnable temporal manifold distance sufficiently capture temporal relationships, we compare it (T3) with squared-time distance (T2). Unlike squared time distances, which treat equal intervals identically regardless of their positions within a window, the learnable temporal manifold distance treats equal intervals at different positions within a window differently. As shown in Table R1, learnable temporal manifold distance reduces MSE by a further 1.17% on average, demonstrating the benefit of learning temporal manifold distance.
  3. Sampling-Conditioned Bandwidth. To examine the contribution of sampling-conditioned bandwidth, we compare T5 with fixed bandwidth (T3), which fixes the length of bandwidth using a hyperparameter, and count-based bandwidth (T4), which learns to adjust the length of bandwidth using the number of nearby observations. As shown in Table R1, our proposed sampling-conditioned bandwidth achieves the best forecasting performance because its anchor-based descriptor captures the local sampling structure.
AuthorsOpenReview ↗

Response to Q2 and W1 — Sensitivity of Cross-Variable Interaction

Thanks for your insightful suggestions. We conduct the following sensitivity analyses.

  1. Number of selected patches. We vary KK from 1 to 32 while keeping the temporal neighbor selection procedure fixed, as shown in Table R2. For K<8K < 8, the selected neighborhood may provide insufficient context, leading to poorer forecasting performance. Once KK reaches 8, performance remains stable across the tested values, showing low sensitivity to the number of selected patches within this range. We recommend K=8K = 8 as a default, as larger values of KK increase the cost of aggregating neighboring patches without yielding consistent gains in forecasting accuracy.

  2. Frequency similarity weighting. We pretrain two variants from scratch on the same corpus with a matched number of optimizer updates, using a reduced pretraining budget due to time constraints. We compare Local + Global frequency weighting with Uniform weighting under the same temporal neighbor selection procedure. Replacing frequency weighting with uniform weights increases both MSE and MAE on all four datasets (Table R3). These results support our choice of Local + Global frequency weighting for aggregating the selected neighbors.

  3. Stability under input changes. We evaluate all four datasets at both forecast horizons on the complete test sets, varying one aspect of the historical input at a time (Table R4).

    • Variable count. We start with the target variable alone and progressively add variables. Adding one auxiliary variable reduces MSE by 7.97% on average, while using all available variables reduces MSE by 9.83% on average relative to using the target variable alone. These results indicate that additional variables generally provide useful forecasting context, although the gains vary across datasets.

    • Sampling density. We retain 75%, 50%, or 25% of historical observations while preserving the first and last observations. Compared with complete historical observations, MSE increases by 1.22%, 3.39%, and 8.15% on average, respectively. These results indicate that the model exhibits low sensitivity to sampling density when the observations are not too sparse.

    • Cross-variable asynchrony. We vary the retained observations to construct low-, medium-, and high-asynchrony inputs, where low asynchrony means that observations across variables occur at similar timestamps, while high asynchrony means that their observation timestamps are more misaligned. From low to high asynchrony, MSE decreases by 0.54% on average. These results indicate low sensitivity to cross-variable asynchrony.

Overall, our model benefits from additional variables and remains relatively stable under moderate historical thinning and changes in cross-variable asynchrony.

Table R2.

KKCESNET ×10−3\times 10^{-3}FNSPID ×10−5\times 10^{-5}Human Activity ×10−4\times 10^{-4}USHCN ×10−3\times 10^{-3}
122.30 / 90.76248.33 / 3014.1943.31 / 401.11993.91 / 482.97
422.02 / 90.29248.12 / 3014.0142.19 / 395.26920.75 / 460.32
822.02 / 90.29248.98 / 3015.7941.79 / 393.35905.82 / 455.19
1622.01 / 90.24249.80 / 3016.1241.45 / 392.10907.77 / 457.05
2421.99 / 90.21249.55 / 3016.8841.70 / 393.24914.81 / 460.11
3221.98 / 90.19250.38 / 3019.4441.89 / 394.20916.92 / 461.27

Table R3.

ConfigurationCESNET ×10−3\times 10^{-3}APTC ×10−5\times 10^{-5}FNSPID ×10−5\times 10^{-5}Human Activity ×10−4\times 10^{-4}
Uniform22.57 / 90.86394.44 / 4653.78352.66 / 3442.3740.37 / 384.32
Local + Global22.19 / 90.48393.63 / 4649.32351.15 / 3434.4840.36 / 384.21

Table R4.

SettingCESNET ×10−3\times 10^{-3}APTC ×10−5\times 10^{-5}FNSPID ×10−5\times 10^{-5}Human Activity ×10−4\times 10^{-4}
1 variable22.49 / 90.60394.30 / 4649.54357.14 / 3484.4843.56 / 393.78
2 variables21.96 / 90.12384.11 / 4590.98253.59 / 3032.3644.45 / 403.56
4 variables22.05 / 90.28—250.98 / 3026.8242.98 / 399.08
All variables, 100% history22.01 / 90.24385.30 / 4597.09249.80 / 3016.1241.45 / 392.10
75% history21.95 / 90.11388.30 / 4615.41255.49 / 3042.6142.32 / 396.53
50% history22.15 / 90.44393.76 / 4647.61261.71 / 3085.4843.92 / 403.10
25% history22.68 / 91.21409.59 / 4737.97273.61 / 3128.3247.13 / 417.65
Low asynchrony22.42 / 90.80393.47 / 4645.57269.90 / 3116.5343.77 / 404.00
Medium asynchrony22.41 / 90.75393.51 / 4645.84266.00 / 3099.2943.43 / 402.71
High asynchrony22.41 / 90.78393.55 / 4646.09264.52 / 3094.8143.71 / 403.07
AuthorsOpenReview ↗

Response to Q3 and W2 — Input adaptation for regular-input baselines

Thanks for your insightful suggestions. We describe how regular-input foundation models receive irregular observations and compare interpolation methods in Appendix B.3. We now extend this comparison with additional experimental results.

Input preprocessing. As described in Algorithm 1 in Appendix A.3, we map each historical sample onto a uniformly spaced temporal grid between its first and last valid timestamps, using linear interpolation for each variable. QiYao-I and MIRA directly use the original irregular timestamps and masks without interpolation. All models are evaluated at the same valid future timestamps, using median forecasts where applicable.

Comparison with alternative input adaptation methods. We further compare four input adaptation strategies: (1) Linear interpolation, which resamples irregular observations onto a uniform grid; (2) BayOTIDE [1], a state-of-the-art imputation method; (3) Binning + Native Mask, which groups observations into fixed-width temporal bins and preserves empty bins through the models' native masking mechanisms; and (4) Binning + TS-ICL [2], which employs TS-ICL for imputation after binning. As shown in Table R5, more sophisticated adaptation strategies do not consistently improve forecasting performance over linear interpolation. These results suggest that simple linear interpolation is already an effective adaptation strategy for regular-input foundation models in our evaluation setting, and more complex preprocessing does not necessarily lead to better forecasting accuracy.

Table R5. Effect of input adaptation on zero-shot forecasting. Entries are MSE / MAE, averaged equally over the two forecast horizons shown for each dataset.

Input adaptationModelCESNET (×10−3\times 10^{-3})FNSPID (×10−5\times 10^{-5})Human Activity (×10−4\times 10^{-4})
LinearChronos-222.00 / 90.94311.00 / 3253.6042.43 / 386.46
LinearToto-222.41 / 92.64280.88 / 3027.2742.95 / 379.01
LinearMoirai-222.45 / 91.02381.45 / 3592.5048.06 / 403.01
BayOTIDEChronos-222.67 / 91.961377.25 / 8382.05240.41 / 1242.97
BayOTIDEToto-228.12 / 99.812877.98 / 12947.19203.70 / 1162.71
BayOTIDEMoirai-222.28 / 91.84783.18 / 6950.05114.40 / 874.15
Binning + Native MaskChronos-225.44 / 95.88337.33 / 3341.2050.40 / 447.23
Binning + Native MaskToto-287.72 / 188.157009.04 / 20236.592166.49 / 4414.08
Binning + Native MaskMoirai-222.04 / 90.30268.58 / 3345.7452.34 / 463.48
Binning + TS-ICLChronos-222.11 / 91.25310.71 / 3230.6542.27 / 396.35
Binning + TS-ICLToto-222.29 / 91.68318.63 / 3249.5642.32 / 393.46
Binning + TS-ICLMoirai-222.05 / 90.58408.24 / 3685.3643.21 / 384.12
Direct irregular inputQiYao-I22.01 / 90.24249.80 / 3016.1241.45 / 392.10

[1] Fang S, Wen Q, Luo Y, et al. BayOTIDE: Bayesian Online Multivariate Time Series Imputation with Functional Decomposition. arXiv preprint arXiv:2308.14906, 2023.

[2] Naour E L, Nabil T, Petralia A. TS-ICL: A Flexible Time-Indexed Foundation Model for Time Series via In-Context Learning. arXiv preprint arXiv:2606.05878, 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.