acceptodds
Under review as a conference paper at ICLR 2027

Unicorn: Universal Correlation Modeling for High-Dimensional Time Series Forecasting

Abstract

Modern time series architectures face a fundamental trade-off: channel-independent models scale well across datasets but ignore inter-channel dependencies, while many channel-dependent models learn correlations within individual datasets and face computational or transfer challenges when channel sets change. To bridge this gap, we introduce Unicorn (Universal Correlation Network), a framework for scalable, multi-dataset pretraining on high-dimensional time series. At the core of Unicorn is a continuous prototype bank that decouples correlation modeling from specific channel identities. By routing heterogeneous channels through a shared latent space, Unicorn learns identity-agnostic, reusable interaction patterns with linear interaction cost in the number of channels, enabling joint pretraining across different channel sets. Experiments on financial and public forecasting benchmarks show competitive forecasting accuracy and strong data efficiency during finetuning, offering a scalable path toward multivariate time series foundation models.

Primary area:
learning on time series and dynamical systems
Keywords:
High-dimensional time series forecasting, cross-domain pretraining

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eQtiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Unicorn addresses the challenge of modeling cross-channel dependencies in high-dimensional multivariate time series while maintaining the scalability and transferability of channel-independent forecasting models. The key idea is to introduce a set of learnable latent prototypes that mediate interactions between channels, thereby avoiding explicit channel-to-channel modeling. In addition, spectral global guidance is incorporated to provide domain-invariant temporal information. The resulting prototype-mediated interaction mechanism reduces the computational cost of correlation modeling and enables pretraining across datasets with different numbers of channels. Extensive experiments validate Unicorn. Pretrained on multi‑market stock data and fine‑tuned on held‑out equity datasets, it outperforms CI and CD baselines in IC/RankIC. It also achieves competitive MSE/MAE on standard multivariate benchmarks under scratch‑training and financial‑pretraining‑finetuning settings. Comprehensive supplementary tests confirm its strengths in structure preservation, few‑shot robustness, and hardware efficiency.

Critical strengths
  1. Modeling cross‑channel dependencies is a major open challenge in multivariate forecasting. Channel‑dependent models capture variable interactions but suffer poor scalability and transferability under changing channel sets. Channel‑independent models scale well for pretraining yet discard cross‑channel signals. Unicorn aims to bridge this gap, combining the scalability of CI models with the correlation‑modeling capacity of CD approaches, which is highly relevant for large‑scale time‑series foundation models.
  2. The core Channel→Prototype→Channel design offers a reasonable alternative to explicit pairwise channel‑to‑channel attention. Instead of learning a full $ C×CC\times C$ interaction matrix, it employs $ K≪CK\ll C$ latent prototypes as a low‑rank bottleneck for channel interactions, reducing computation from O(C2)O(C^2) to O(CK)O(CK). This design is especially beneficial for high‑dimensional datasets with thousands of variables.
  3. Prototypes are decoupled from concrete channel identities. This makes the architecture compatible with datasets of varying channel counts. Cross‑dataset pretraining experiments provide meaningful evidence that the learned representations can transfer across different channel configurations, constituting a clear advantage over most traditional channel‑dependent models.
  4. Evaluations cover both cross‑domain pretraining and high‑dimensional forecasting tasks on large‑channel‑count datasets. Ablation studies validate the contributions of both prototype‑mediated interaction and spectral global guidance. Besides standard forecasting metrics, the paper includes valuable supplementary evidence: prototype freeze‑vs‑reinit controlled experiments, structure‑aware metrics (Corr‑MAE, ACF‑MAE, PSD‑L1), few‑shot adaptation tests, and hardware‑level efficiency profiling, all of which deepen our understanding of model behavior.
  5. The paper is well‑written; key properties such as permutation equivariance and rich implementation details are documented in appendices.
Critical weaknesses
  1. The paper suggests that learned prototypes capture universal cross‑channel correlation patterns. Since prototypes are learned from temporal‑spectral features, I am wondering whether they encode genuine correlation structures or merely cluster channels that share similar temporal or spectral signatures. Representation similarity is not equivalent to correlation similarity; for example, two sine‑wave time series can have nearly identical spectra yet be strongly anti‑correlated. Could the authors add diagnostic analysis to examine whether prototype assignments align with actual pairwise/partial correlation, beyond just temporal‑spectral similarity?
  2. Strong transfer results are mostly observed within financial‑related domains, while performance gains remain modest when transferring from finance to dissimilar domains such as traffic or energy. To what extent do the results support truly universal correlation structures versus more limited transferable interaction primitives? I suggest the authors consider softening the “universal” wording and clarifying the practical scope of cross‑domain transfer in the manuscript.
  3. Prototype‑based modeling brings empirical gains, but it is unclear whether these improvements originate from transferable knowledge stored inside pretrained prototypes or simply come from architectural inductive bias or implicit regularization. Would it be possible to run a controlled experiment: freeze pretrained prototypes when transferring to unseen target datasets, and compare against settings with randomly initialized prototypes and fully fine‑tuned prototypes? This could help isolate the actual contribution of pretrained prototype knowledge.
  4. Current ablations mix frequency‑domain information, global aggregation, and extra model capacity together. Can the authors further disentangle how much performance gain truly comes from the frequency‑oriented spectral inductive bias itself? The Corr‑MAE metric shows good output‑level correlation matching. Does this necessarily imply that the internal prototype mechanism has learned meaningful correlation structures, or could this outcome be achieved via alternative internal representations?
  5. TimesFM‑3.0 became publicly available before the submission deadline and supports native variate‑attention for multivariate forecasting. Would it be feasible to include it as a supplementary baseline, reporting both forecasting metrics and hardware overhead, especially under high‑channel‑count conditions?
Questions for authors

Please refer to the Critical Weaknesses section.

1 comment

Thank you for reading our paper so carefully and for your thoughtful questions about what the prototypes learn.

W1. Assignments and correlations

At submission, we had not tested whether prototype assignments align with empirical pairwise or partial correlations. In the reviewer’s example, a sinusoid and its negative have identical Fourier magnitudes and correlation −1-1. Unicorn retains the original temporal features alongside the guidance features (Eqs. 4–5), so the two signals can still have different inputs to the bank. The nonnegative attention weights in Eqs. (6)–(7) describe how channels exchange features through the prototypes.

Table 7 tests whether the forecasts preserve channel co-movement. Corr-MAE averages the absolute difference between the signed pairwise Pearson correlations of the forecast and target over all off-diagonal channel pairs.

Forecast Corr-MAE (lower is better)TrafficCrime-ChicagoElectricity
TimeBridge0.1080.1610.081
Unicorn0.1010.1340.074

Unicorn’s forecasts have lower correlation error on all three datasets. Different internal representations can produce the same output correlations. Table 9 gives a separate observation about the assignments themselves. Channels grouped by their dominant prototype have lower within-group variance in period, volatility, and lag-one autocorrelation than random groups of the same sizes.

W2. Scope of transfer

In Appendix E.1, “universal” refers to the prototype parameters shared across datasets. The current channels determine the assignments, summaries, and resulting interactions. The observed gain from transfer depends on the target.

Table 1 compares target-only training with financial pretraining followed by target finetuning. Raw IC rises from 0.0185 to 0.0257 on A-share and from 0.0139 to 0.0238 on NASDAQ-100. Both markets were represented in pretraining and evaluated on a later period (Appendix D).

Pretraining before target training (Table 10)Traffic (MSE / MAE)Electricity (MSE / MAE)
None, train from scratch on the target0.343 / 0.2390.120 / 0.212
Financial data only0.341 / 0.2380.121 / 0.211
Financial data and the target training split0.335 / 0.2340.119 / 0.205

After target finetuning, financial-only pretraining gives small gains on Traffic. On Electricity, MAE falls from 0.212 to 0.211 while MSE rises from 0.120 to 0.121. Table 2 also shows a larger gain on ECL-Daily, where MSE falls from 0.265 to 0.241. The lowest errors in Table 10 come from two separate runs, each combining finance with the respective target training split. Only the financial-only pretraining row measures transfer without prior exposure to that public dataset.

W3. Freezing pretrained prototypes

We had not tested frozen prototypes on a dataset absent from pretraining at submission. Table 2 uses full-model finetuning on those public targets. Table 8 compares the three prototype settings on A-share and NASDAQ-100, both in markets represented during pretraining.

Raw IC (Table 8)Full finetuningFreeze pretrained bankRandomly reinitialize and finetune bank
A-share0.02570.02510.0203
NASDAQ-1000.02380.02320.0181

All three conditions use the same architecture, non-prototype weights from the same checkpoint, target data, finetuning budget, and checkpoint-selection rule. On each market, freezing the pretrained bank costs only 0.0006 raw IC relative to full finetuning. A randomly initialized bank gives lower IC on both markets even though it remains trainable and Eq. (9) still regularizes it toward the original pretrained bank. The reinitialized bank therefore still has access to the pretrained reference during finetuning. Its lower IC shows that starting from the learned values helps under the same architecture and adaptation objective.

W4. Spectral guidance and Corr-MAE

We had not isolated the contribution of the periodic features at submission. The without-guidance variant in Figure 3 removes the periodic features together with global aggregation and its associated parameters. Figure 3 separately tests the spectral loss by dropping the Fourier term from the training objective while keeping all trainable layers.

Figure 3 ablationWhat changes during trainingWhat the result shows
Without Spectral GuidanceRemove the FAN descriptors and the pathway that pools and returns global context.Forecasting errors generally increase when the complete guidance module is removed.
Without Spectral LossTrain the same architecture using only the time-domain objective.MSE and MAE increase on all four evaluated datasets when Fourier supervision is removed.

Table 7 measures correlations in the forecasts. Corr-MAE cannot attribute those correlations to the periodic features.

Reviewer_zvvKOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Unicorn, a universal correlation modeling framework for multivariate time series forecasting. The key idea is to learn transferable channel interaction patterns across different datasets through a prototype-based interaction mechanism. Specifically, the model first extracts channel-wise temporal representations, then introduces a spectral-guided module to capture global information, and employs a prototype interaction module to aggregate and redistribute cross-channel information via a small set of learnable prototypes. The authors further introduce a spectral-temporal forecasting objective and a prototype regularization strategy for cross-domain adaptation. Extensive experiments on multiple multivariate time series benchmarks demonstrate the effectiveness of Unicorn compared with existing forecasting approaches.

Critical strengths
  1. The paper addresses an important problem in multivariate time series forecasting, namely how to effectively model cross-variable dependencies while improving the transferability across different datasets.
  2. The proposed framework is comprehensive and integrates several components, including channel-wise temporal modeling, global guidance, and prototype-based interaction, forming a unified forecasting pipeline.
  3. The authors conduct extensive experiments on multiple benchmark datasets and provide comparisons with a variety of existing forecasting methods.
Critical weaknesses
  1. The proposed prototype interaction mechanism appears closely related to existing router-based interaction approaches in multivariate time series forecasting. In particular, the idea of using a small number of learnable latent vectors to aggregate information from multiple variables and then redistribute the information has been explored in previous works such as Crossformer and Phaseformer. Therefore, the novelty of introducing prototypes as universal correlation representations is not sufficiently. Moreover, the Univariate Temporal Modeling component directly adopts TimeBridge as the temporal backbone rather than the commonly used patch-based Transformer architectures.
  2. The paper claims that the learned prototypes capture universal correlation patterns across datasets. However, the experiments mainly focus on forecasting performance improvements, and there is limited analysis demonstrating that the prototypes actually represent transferable correlation structures rather than dataset-specific latent features.
  3. The provided supplementary code does not contain sufficient details and support for reproducing the experimental pipeline described in the paper. According to the provided data loader, if the stated 70/10/20 temporal split is used, the validation portions of Wiki-People and Traffic-Daily contain only approximately 55 and 74 timestamps, respectively, which appear insufficient to construct validation samples with the reported forecasting horizon of 96 steps. In addition, the scripts directory is empty, and dataset-specific execution commands and complete hyperparameter configurations are not provided. Furthermore, the provided training entry does not clearly implement the full workflow described in the paper, including multi-market joint pretraining, loading pretrained checkpoints, and target-domain finetuning. These discrepancies make it difficult to verify whether the reported results were obtained under the experimental settings described in the paper.
Questions for authors
  1. The proposed channel-prototype interaction follows a similar aggregate-and-redistribute paradigm to previous router-based approaches such as Crossformer and Phaseformer. Could the authors clarify the fundamental differences between the proposed continuous prototype bank and existing router/latent interaction mechanisms? Moreover, given that the temporal modeling component adopts TimeBridge as the backbone, could the authors further explain why a simple patch-based Transformer architecture is insufficient for this framework and what specific limitations are addressed by the adopted design?
  2. Could the authors clarify the exact data split and validation procedure used for the reported results? In particular, for short datasets such as Wiki-People and Traffic-Daily, the provided implementation appears to split the raw time series into separate temporal segments before constructing forecasting samples, rather than constructing sliding-window samples first. Under this procedure, the validation segments may be insufficient to support the reported 96-step forecasting horizon.
1 comment

We appreciate the time you took to examine our architecture and the reproducibility of our experiments.

W1/Q1. Routers and temporal backbone

Routers. All three designs aggregate and redistribute features. Unicorn’s specific aim is to learn a small set of shared interaction patterns for sparse predictive dependencies among many channels, and to reuse those patterns when channel identities and counts change (Appendix E.1).

MethodWhat interacts?How the routing works
CrossformerVariables at a given temporal segmentSegment-specific routers gather and redistribute features across variables. The routers are independent of channel count. The released model also has channel-indexed positional embeddings.
PhaseFormerPhase tokens within one seriesRouters gather information from different phases and return it to the phase tokens.
UnicornChannels at each long-term indexSpectral Global Guidance supplies shared context. The same KK prototypes pool features from the current channels, and each channel retrieves from those summaries.

Unicorn shares the encoder, guidance module, bank, and forecasting head across channel sets. The prototypes attend over channels to form summaries (Eq. 6), then each channel attends over those summaries (Eq. 7). Reordering the input channels therefore reorders the forecasts in the same way (Appendix E.1). Because none of the shared parameters is indexed by channel identity, we could pretrain one model across three stock markets and finetune it on panels of 587 and 88 stocks (Section 5.1).

Temporal backbone. Unicorn already represents each channel as a sequence of patches. We chose TimeBridge’s Integrated Attention to handle short-term nonstationarity within a channel. Temporal downsampling reduces the number of time indices passed to the cross-channel modules (Section 4.1). We had not compared this backbone with a standard patch Transformer at submission. Figure 3 holds the temporal backbone fixed. Replacing prototype routing with direct channel interaction increases MSE and MAE on all four datasets. This comparison tests the choice of interaction module within the adopted backbone.

W2. Prototype reuse across markets

In Table 8, all three conditions start non-prototype weights from the same checkpoint and use the same target data, split, finetuning budget, and checkpoint-selection rule.

Prototype treatmentA-share raw ICNASDAQ-100 raw IC
Finetune the pretrained bank and the rest of the model0.02570.0238
Freeze the pretrained bank and finetune the rest of the model0.02510.0232
Randomly reinitialize the bank and finetune the whole model0.02030.0181

Freezing the pretrained bank lowers IC by 0.0006 on each target relative to full finetuning. Random reinitialization lowers it by 0.0054 on A-share and 0.0057 on NASDAQ-100, even though Eq. (9) still regularizes the bank toward its original pretrained values. The pretrained starting values thus matter under this adaptation protocol.

Table 9 examines how the bank is used. The two markets’ average prototype-usage profiles have cosine similarity 0.87. Channels grouped by their dominant prototype also have lower within-group variance than size-matched random groups in dominant period, volatility, and lag-one autocorrelation (ratios 0.68, 0.74, and 0.79). Both markets were represented in pretraining. Similar average usage alone does not establish reuse. Table 8 shows that retaining the bank also preserves forecasting performance during adaptation.

W3/Q2. Short-dataset validation

We split each raw series chronologically before constructing sliding windows. For Wiki-People and Traffic-Daily, an input that crosses the training boundary draws its earlier observations from the end of training. This supplies the input history and keeps the forecast horizon unchanged. The Wiki-People input construction follows STHD. Test forecasts are scored against complete target windows from the test segment.

As the reviewer notes, under the reported split and horizon, the supplied loader cannot form a full target window within either validation segment. Appendix D gives the checkpoint-selection criterion (validation MSE), but our submission does not document the target windows used to compute it for these two datasets.

W3. Supplementary code

We could not include the datasets with the submission, so we simplified the code provided for review. We recognize that this makes it difficult to reproduce the full experiments from the supplement alone.

Appendix D describes the protocol and Table 6 gives the common hyperparameters. We plan to organize the code and experiment entry points for public release.

Reviewer_GsEEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Unicorn, a framework for high-dimensional time-series forecasting that aims to learn transferable cross-channel dependencies across datasets with different channel identities and sizes. Unicorn introduces a shared continuous prototype bank to model channel interactions through a low-rank routing mechanism, together with spectral global guidance to capture periodic patterns. Experiments demonstrate promising cross-market transfer performance, competitive forecasting accuracy, and robustness to channel subsampling and few-shot finetuning.

Critical strengths

The channel-independent prototype bank provides an effective way to model transferable cross-channel dependencies while reducing computational complexity from quadratic to linear in the number of channels.

The combination of prototype-based interaction and spectral global guidance is well motivated, and the ablation studies demonstrate the contributions of the main components.

The experiments cover cross-market transfer, standard forecasting benchmarks, channel scalability, and few-shot finetuning. The strong transfer results are particularly encouraging for high-dimensional forecasting applications.

Critical weaknesses

Although computational scalability is a central motivation, the paper provides limited empirical comparisons of training time, inference latency, and GPU memory consumption against strong channel-dependent baselines. Also, the paper doesn’t show the performance and efficiency of Unicorn on Time-HD, a high-dimensional time series benchmark.

Some implementation details, particularly the spectral loss formulation and spectral guidance module, are insufficiently explained. Moreover, the low-rank prototype bottleneck may limit the representation of complex channel dependencies, but this limitation is not thoroughly investigated.

Questions for authors
  1. The idea of channel-prototype interaction module that replaces dense C×C attention with a factorized C->K->C pathway is similar to the latent query module in U-Cast. Could you explain the difference and why the Unicorn design is better?

  2. Can the authors provide detailed runtime and peak GPU memory comparisons against dense channel attention models across different channel counts and prototype sizes?

  3. How exactly is the spectral loss computed? Does it compare FFT magnitudes, complex coefficients, or both, and how sensitive are the results to this choice?

  4. How stable are the learned prototype assignments under substantial channel subsampling or domain shifts, particularly when the prototype number (K) remains fixed?

1 comment

Thank you for your careful review and thoughtful questions on scalability.

W1/Q2. Runtime, memory, and Time-HD

At submission, we had not run the requested hardware comparison across channel counts and prototype sizes or evaluated Time-HD. Appendix E.6 reports end-to-end measurements on the full 587-stock A-share panel with an RTX 4090 and K=16K=16 for Unicorn.

ModelTraining (s/epoch)Inference (ms/panel)Peak training memory (GiB)Peak inference memory (GiB)
TimeBridge319.815.21.2
Unicorn227.011.40.9

On this panel, Unicorn reduces inference latency by 28.6% and peak inference memory by 25% relative to TimeBridge. For CC channels and feature width DD, the per-index interaction cost is O(CKD)O(CKD) for prototype routing and O(C2D)O(C^2D) for dense attention (Section 4.5). Unicorn applies the two attention matrices in sequence and never materializes their C×CC\times C product.

W2/Q3. Spectral loss and guidance

Loss. Eq. (8) compares complex Fourier coefficients. For each channel, we take the real-input FFT along the forecast horizon. We average ∣z^−z∣\lvert\hat z-z\rvert over frequency bins, channels, and examples, then combine this term with time-domain MAE using weights α\alpha and 1−α1-\alpha, respectively. For example, coefficients zz and −z-z have equal magnitudes but incur a penalty of 2∣z∣2\lvert z\rvert. We had not tested a magnitude-only loss at submission. Figure 3 shows that removing the spectral term (α=0\alpha=0) increases forecasting error on all four datasets relative to the default (α=0.3\alpha=0.3).

Guidance. In Eqs. (1)–(4), a learned projection of FAN’s sine and cosine features gives each channel a score. A softmax across channels turns the scores into weights at each long-term index. The weighted sum of the original temporal features gives a global context, which is fused back into each channel before prototype routing. The explicit time-axis FFT is used only in the loss.

W2. Rank and prototype count

At time index mm, channel interaction passes through KK summaries, so the routing matrix BmAmB_m A_m has rank at most KK (Section 4.5). The bound applies to a single routing step for a given input. Both routing matrices are recomputed when the input changes. The full predictor also uses channel-wise residual connections and nonlinear transformations.

Table 3 varies only the prototype count. Higher raw IC is better.

Target (raw IC)K=2K=2K=4K=4K=8K=8K=16K=16 (default)K=32K=32
A-share0.01870.02140.02490.02570.0259
NASDAQ-1000.01850.02310.02410.02380.0226

Both markets improve from two to eight prototypes. Beyond eight, A-share IC continues to rise, while NASDAQ-100 IC falls. As Section 6 notes, a small bank can limit capacity for complex interactions.

Q1. U-Cast and Unicorn

Both methods use learned queries for channel interaction. Unicorn’s fixed-size bank captures shared predictive patterns and lets datasets with different variables reuse the same interaction parameters.

Design choiceU-CastUnicorn
Channel interactionLearned queries downsample channel representations, then an upsampling stage restores the channel dimension.At each long-term index, KK prototypes pool channel features. Each channel then reads from those summaries.
Size of the learned queriesLevel ℓ\ell has ⌊C/rℓ⌋\lfloor C/r^\ell\rfloor queries for CC channels and reduction factor rr.The bank keeps the same KK rows across datasets. Routing weights and summaries are computed from the current input.
RegularizationThe full-rank penalty encourages diversity among latent representations.During finetuning, Eq. (9) penalizes changes to the pretrained bank.

We had not compared forecasting accuracy with U-Cast at submission. We used a single bank of 16 learned prototypes to pretrain across three market datasets whose full panels range from 2,418 to 7,036 assets (Section 5.1 and Appendix D).

Q4. Assignment stability

Channel subsampling. At submission, we had not measured assignment changes for individual channels after subsampling. Table 4 reports A-share raw IC after finetuning with all channels. It is 0.0257 with the full panel, 0.0244 with half the channels, and 0.0238 with one quarter, using the same 16 prototypes throughout. Removing channels changes the set over which each prototype normalizes its weights (Eq. 6), so a retained channel can receive a different mixture of summaries.

Target-market adaptation. Table 8 measures assignments directly on the same target inputs before and after finetuning. Mean per-channel L1L_1 assignment drift (Eq. 20) is 0.41 on A-share and 0.47 on NASDAQ-100. Relative prototype drift (Eq. 19) is 0.07 and 0.09 under the regularization in Eq. (9). These measures use different scales. Freezing the pretrained bank lowers raw IC by only 0.0006 on each market relative to full finetuning.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.