- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Unicorn addresses the challenge of modeling cross-channel dependencies in high-dimensional multivariate time series while maintaining the scalability and transferability of channel-independent forecasting models. The key idea is to introduce a set of learnable latent prototypes that mediate interactions between channels, thereby avoiding explicit channel-to-channel modeling. In addition, spectral global guidance is incorporated to provide domain-invariant temporal information. The resulting prototype-mediated interaction mechanism reduces the computational cost of correlation modeling and enables pretraining across datasets with different numbers of channels. Extensive experiments validate Unicorn. Pretrained on multi‑market stock data and fine‑tuned on held‑out equity datasets, it outperforms CI and CD baselines in IC/RankIC. It also achieves competitive MSE/MAE on standard multivariate benchmarks under scratch‑training and financial‑pretraining‑finetuning settings. Comprehensive supplementary tests confirm its strengths in structure preservation, few‑shot robustness, and hardware efficiency.
Critical strengths
- Modeling cross‑channel dependencies is a major open challenge in multivariate forecasting. Channel‑dependent models capture variable interactions but suffer poor scalability and transferability under changing channel sets. Channel‑independent models scale well for pretraining yet discard cross‑channel signals. Unicorn aims to bridge this gap, combining the scalability of CI models with the correlation‑modeling capacity of CD approaches, which is highly relevant for large‑scale time‑series foundation models.
- The core Channel→Prototype→Channel design offers a reasonable alternative to explicit pairwise channel‑to‑channel attention. Instead of learning a full $ $ interaction matrix, it employs $ $ latent prototypes as a low‑rank bottleneck for channel interactions, reducing computation from to . This design is especially beneficial for high‑dimensional datasets with thousands of variables.
- Prototypes are decoupled from concrete channel identities. This makes the architecture compatible with datasets of varying channel counts. Cross‑dataset pretraining experiments provide meaningful evidence that the learned representations can transfer across different channel configurations, constituting a clear advantage over most traditional channel‑dependent models.
- Evaluations cover both cross‑domain pretraining and high‑dimensional forecasting tasks on large‑channel‑count datasets. Ablation studies validate the contributions of both prototype‑mediated interaction and spectral global guidance. Besides standard forecasting metrics, the paper includes valuable supplementary evidence: prototype freeze‑vs‑reinit controlled experiments, structure‑aware metrics (Corr‑MAE, ACF‑MAE, PSD‑L1), few‑shot adaptation tests, and hardware‑level efficiency profiling, all of which deepen our understanding of model behavior.
- The paper is well‑written; key properties such as permutation equivariance and rich implementation details are documented in appendices.
Critical weaknesses
- The paper suggests that learned prototypes capture universal cross‑channel correlation patterns. Since prototypes are learned from temporal‑spectral features, I am wondering whether they encode genuine correlation structures or merely cluster channels that share similar temporal or spectral signatures. Representation similarity is not equivalent to correlation similarity; for example, two sine‑wave time series can have nearly identical spectra yet be strongly anti‑correlated. Could the authors add diagnostic analysis to examine whether prototype assignments align with actual pairwise/partial correlation, beyond just temporal‑spectral similarity?
- Strong transfer results are mostly observed within financial‑related domains, while performance gains remain modest when transferring from finance to dissimilar domains such as traffic or energy. To what extent do the results support truly universal correlation structures versus more limited transferable interaction primitives? I suggest the authors consider softening the “universal” wording and clarifying the practical scope of cross‑domain transfer in the manuscript.
- Prototype‑based modeling brings empirical gains, but it is unclear whether these improvements originate from transferable knowledge stored inside pretrained prototypes or simply come from architectural inductive bias or implicit regularization. Would it be possible to run a controlled experiment: freeze pretrained prototypes when transferring to unseen target datasets, and compare against settings with randomly initialized prototypes and fully fine‑tuned prototypes? This could help isolate the actual contribution of pretrained prototype knowledge.
- Current ablations mix frequency‑domain information, global aggregation, and extra model capacity together. Can the authors further disentangle how much performance gain truly comes from the frequency‑oriented spectral inductive bias itself? The Corr‑MAE metric shows good output‑level correlation matching. Does this necessarily imply that the internal prototype mechanism has learned meaningful correlation structures, or could this outcome be achieved via alternative internal representations?
- TimesFM‑3.0 became publicly available before the submission deadline and supports native variate‑attention for multivariate forecasting. Would it be feasible to include it as a supplementary baseline, reporting both forecasting metrics and hardware overhead, especially under high‑channel‑count conditions?
Questions for authors
Please refer to the Critical Weaknesses section.
Thank you for reading our paper so carefully and for your thoughtful questions about what the prototypes learn.
W1. Assignments and correlations
At submission, we had not tested whether prototype assignments align with empirical pairwise or partial correlations. In the reviewer’s example, a sinusoid and its negative have identical Fourier magnitudes and correlation −1. Unicorn retains the original temporal features alongside the guidance features (Eqs. 4–5), so the two signals can still have different inputs to the bank. The nonnegative attention weights in Eqs. (6)–(7) describe how channels exchange features through the prototypes.
Table 7 tests whether the forecasts preserve channel co-movement. Corr-MAE averages the absolute difference between the signed pairwise Pearson correlations of the forecast and target over all off-diagonal channel pairs.
Unicorn’s forecasts have lower correlation error on all three datasets. Different internal representations can produce the same output correlations. Table 9 gives a separate observation about the assignments themselves. Channels grouped by their dominant prototype have lower within-group variance in period, volatility, and lag-one autocorrelation than random groups of the same sizes.
W2. Scope of transfer
In Appendix E.1, “universal” refers to the prototype parameters shared across datasets. The current channels determine the assignments, summaries, and resulting interactions. The observed gain from transfer depends on the target.
Table 1 compares target-only training with financial pretraining followed by target finetuning. Raw IC rises from 0.0185 to 0.0257 on A-share and from 0.0139 to 0.0238 on NASDAQ-100. Both markets were represented in pretraining and evaluated on a later period (Appendix D).
After target finetuning, financial-only pretraining gives small gains on Traffic. On Electricity, MAE falls from 0.212 to 0.211 while MSE rises from 0.120 to 0.121. Table 2 also shows a larger gain on ECL-Daily, where MSE falls from 0.265 to 0.241. The lowest errors in Table 10 come from two separate runs, each combining finance with the respective target training split. Only the financial-only pretraining row measures transfer without prior exposure to that public dataset.
W3. Freezing pretrained prototypes
We had not tested frozen prototypes on a dataset absent from pretraining at submission. Table 2 uses full-model finetuning on those public targets. Table 8 compares the three prototype settings on A-share and NASDAQ-100, both in markets represented during pretraining.
All three conditions use the same architecture, non-prototype weights from the same checkpoint, target data, finetuning budget, and checkpoint-selection rule. On each market, freezing the pretrained bank costs only 0.0006 raw IC relative to full finetuning. A randomly initialized bank gives lower IC on both markets even though it remains trainable and Eq. (9) still regularizes it toward the original pretrained bank. The reinitialized bank therefore still has access to the pretrained reference during finetuning. Its lower IC shows that starting from the learned values helps under the same architecture and adaptation objective.
W4. Spectral guidance and Corr-MAE
We had not isolated the contribution of the periodic features at submission. The without-guidance variant in Figure 3 removes the periodic features together with global aggregation and its associated parameters. Figure 3 separately tests the spectral loss by dropping the Fourier term from the training objective while keeping all trainable layers.
Table 7 measures correlations in the forecasts. Corr-MAE cannot attribute those correlations to the periodic features.