- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper addresses multivariate long-term forecasting with a frozen ImageNet MAE, following the VisionTS imaging pipeline. It identifies two problems: channel-independent imaging ignores cross-variable dependencies, and the two image axes (across-period and within-period) carry temporal meaning that a natural-image MAE does not model. MUSE adds two trainable branches around the frozen MAE. VCR uses a TimeXer-style channel token per variable, inter-variable self-attention, and an additive per-variable correction before the encoder. TPR inserts AdaptFormer-style bottleneck adapters with axial temporal and circular periodic attention after the encoder blocks. The two branch forecasts are fused by a scalar gate. Experiments cover 10 TFB datasets at 4 horizons against 8 baselines (Table 1, Table 4), with an ablation on 4 datasets at H = 720 (Table 2) and a sensitivity study (Fig. 3).
Critical strengths
- The trade-off between channel-independent imaging (VisionTS, DMMV) and joint imaging (VisionTS++, Time-VLM) is stated clearly, and adding cross-variable context outside the frozen backbone is a reasonable way to keep the pre-trained representation intact.
- TPR's directional bias on the temporal axis and circular bias on the phase axis (Eqs. 9-10) are a sensible structural prior for period-folded images.
- The method is specified in enough detail to reimplement (Alg. 1, App. B.2), code is provided, the tables are internally consistent, and the evaluation spans 10 TFB datasets.
Critical weaknesses
- The "structural semantic gap" is asserted but not demonstrated. The paper's motivation is that a frozen natural-image MAE cannot model the temporal meaning of the two image axes, but no analysis (attention maps, probing, or a failure case of plain VisionTS) shows this gap or shows that TPR closes it. A related conceptual question is left open: once trainable adapters are added inside the backbone, what does the frozen vision prior still contribute compared with a strong forecaster trained from scratch?
- The cross-variable claim is not tested directly. VCR adds one additive vector per variable, broadcast over all patches (Eq. 5). It is not clear that this static correction can represent lagged or time-varying dependencies, which are what multivariate methods usually target. On Electricity (V = 321), where dependencies should matter most, MUSE is roughly on par with DMMV-A. A channel-shuffle test or an analysis of the inter-variable attention would make the claim concrete.
- Positioning and novelty are limited. VCR follows TimeXer, TPR follows AdaptFormer with axial attention, and the method combines them with a prediction-level gate. The closest related methods (VisionTS++, Time-VLM, TimeXer) are not compared, and only full-shot results are reported even though transfer is the usual reason for using a frozen pre-trained backbone. The modules are called "lightweight", but each branch runs the MAE, and no cost numbers are given.
Questions for authors
- What evidence shows that the frozen MAE fails to model the across-period and within-period axes, and that TPR fixes this? Can you show attention patterns or a probing result?
- What does the frozen vision backbone contribute beyond VCR + TPR? For example, how does MUSE perform with a randomly initialized ViT of the same size, or compared with TimeXer?
- Can VCR's per-variable additive correction capture lagged cross-variable dependencies? How does performance change if the variable order is shuffled or the inter-variable attention is removed?
7 comments
Response to Reviewer J9p9 (Part II)
W1.2 & W3 & Q2: The added value of the frozen visual prior remains unclear, comparisons with VisionTS++, Time-VLM, and TimeXer are missing, and the full-shot setting and the cost of running both branches need clarification.
We thank the reviewer for raising specific questions about the contribution of the frozen visual prior, comparisons with related methods, and MUSE’s training setting and parameter overhead.
Through masked reconstruction pretraining on a large collection of natural images, MAE learns rich visual priors about local shapes, spatial layouts, and recurring patterns. It learns to use structural cues from visible patches to reconstruct masked regions. When a time series is arranged into a 2D image by period, local correlations, trends, and recurring patterns can appear as spatial structures. MUSE transfers MAE’s pretrained reconstruction capability to forecasting by reconstructing the masked future region from the visible historical region. VCR and TPR adapt the pretrained representations to cross-variable context and temporal–periodic dependencies, respectively.
PatchTST, for example, learns forecasting mappings directly in the numerical domain. As shown in and , MUSE achieves lower average MSE and MAE than PatchTST on all 10 datasets. These results support the effectiveness of combining a frozen visual backbone with adaptation tailored to time-series dependencies.
We further compared MUSE with the three methods suggested by the reviewer, VisionTS++, TimeXer, and Time-VLM, on six shared datasets. Each cell in the two tables below reports MSE / MAE averaged over four forecast horizons (96, 192, 336, and 720). Lower is better, and the best value for each metric is in bold.
VisionTS++ targets general-purpose time-series forecasting. It continues pretraining on LOTSA, which contains over 231 billion observations, and updates all backbone parameters. MUSE focuses on adapting a frozen visual backbone and uses the target training set to update only the newly introduced adaptation parameters. Given these differences in research objectives and training protocols, we omitted VisionTS++ from the main comparison in the original paper. The table below compares the two methods using MAE-Base in both cases.
Comparison of MUSE and VisionTS++ on six shared datasets.
| Dataset | MUSE | VisionTS++ |
|---|---|---|
| ETTh1 | 0.384 / 0.405 | 0.402 / 0.416 |
| ETTh2 | 0.325 / 0.372 | 0.333 / 0.370 |
| ETTm1 | 0.330 / 0.362 | 0.360 / 0.372 |
| ETTm2 | 0.245 / 0.309 | 0.244 / 0.298 |
| Electricity | 0.157 / 0.251 | 0.184 / 0.265 |
| Weather | 0.216 / 0.261 | 0.222 / 0.241 |
As shown in the table, MUSE achieves lower average MSE on five of the six datasets and lower values for both metrics on ETTh1, ETTm1, and Electricity. The next table presents the comparison with TimeXer and Time-VLM on the same six datasets.
Comparison of MUSE, TimeXer, and Time-VLM on six shared datasets.
| Dataset | MUSE | TimeXer | Time-VLM |
|---|---|---|---|
| ETTh1 | 0.384 / 0.405 | 0.437 / 0.437 | 0.405 / 0.420 |
| ETTh2 | 0.325 / 0.372 | 0.367 / 0.396 | 0.341 / 0.391 |
| ETTm1 | 0.330 / 0.362 | 0.382 / 0.397 | 0.350 / 0.377 |
| ETTm2 | 0.245 / 0.309 | 0.274 / 0.322 | 0.248 / 0.311 |
| Electricity | 0.157 / 0.251 | 0.171 / 0.270 | 0.172 / 0.273 |
| Weather | 0.216 / 0.261 | 0.241 / 0.271 | 0.224 / 0.263 |
As shown in the table, MUSE achieves lower average MSE and MAE than both TimeXer and Time-VLM on all six datasets.
We use the full-shot setting to evaluate how well pretrained visual representations transfer to time-series forecasting when the entire target training set is available for adaptation. The MAE encoder and decoder remain frozen throughout training, and we update only the newly introduced adaptation parameters. The frozen MAE contains 111.908M parameters, and MUSE adds 1.147M trainable parameters, equivalent to 1.03% of the MAE parameter count. Approximately 98.99% of the full model’s parameters are frozen. In the paper, “lightweight” refers specifically to the trainable parameter overhead of adaptation. We will include these baseline comparisons and parameter counts in the final version and clarify the role of the frozen visual prior, the full-shot training setting, and what we mean by “lightweight.”
Response to Reviewer J9p9 (Part III)
W2 & Q3: It remains unclear whether VCR can capture lagged or time-varying cross-variable dependencies. Its contribution to cross-variable modeling needs direct validation through channel shuffling, removal of inter-variable attention, or analysis of attention weights.
We thank the reviewer for pointing out a limitation of VCR and suggesting that we directly test the role of cross-variable attention. We apologize for the lack of clarity about this limitation in the original paper. As shown in , VCR generates one correction vector per variable and adds it to all patches of that variable within the same historical window. Consequently, VCR cannot capture lagged or time-varying cross-variable dependencies.
On Electricity, which has 321 variables, MUSE and DMMV-A perform comparably. reports average MSE / MAE of 0.157 / 0.251 for MUSE and 0.158 / 0.248 for DMMV-A. We interpret these results using two dataset characteristics (stationarity and transition) from TFB [1].
Definition 1 (Stationarity). In a weakly stationary series, the mean and variance are constant, the variance is finite, and the covariance between any two observations depends only on their time lag. For any shift () that keeps both indices within the series,
TFB assesses stationarity using the augmented Dickey–Fuller (ADF) test and defines the dataset-level Stationarity value as the mean of the ADF (p)-values across variables.
Definition 2 (Transition). Transition is the trace of the covariance matrix associated with the transition matrix for a three-symbol alphabet. It characterizes identifiable temporal patterns such as trends and periodicity. Lower values correspond to clearer temporal regularities, while higher values correspond to less pronounced patterns.
Electricity has a Stationarity value of 0.00515, indicating strong evidence of stationarity, and a Transition value of 0.01049, indicating that trends and periodic patterns are readily identifiable. These clear within-variable temporal and periodic patterns are well suited to TPR’s design. By modeling within-variable dependencies along the historical time and phase axes, respectively, TPR plays a more important role in MUSE’s forecasts on Electricity.
To directly assess the contribution of cross-variable attention to forecasting, we conducted an intra-variable-only VCR ablation. reports ablation results on ETTh2, Weather, AQWan, and ZafNoo at a forecast horizon of . We evaluated this control on the same datasets and forecast horizon. In the full model, VCR uses cross-variable interaction to generate a correction vector for each variable. In this ablation, we remove the inter-variable multi-head self-attention in .
Comparison of the full model and the intra-variable-only VCR control. Each cell reports MSE / MAE. Lower is better, and the best result for each metric is in bold.
| Dataset | Full Model | VCR intra-variable only |
|---|---|---|
| ETTh2 | 0.373 / 0.416 | 0.376 / 0.419 |
| Weather | 0.299 / 0.326 | 0.303 / 0.328 |
| AQWan | 0.883 / 0.526 | 0.888 / 0.530 |
| ZafNoo | 0.564 / 0.468 | 0.570 / 0.471 |
As shown in the table, the full model achieves lower MSE and MAE than the intra-variable-only control on all four datasets. This indicates that the historical representations of other variables can provide additional predictive information for each variable. VCR uses cross-variable attention to incorporate this information into the variable-level correction vectors, improving forecasting performance.
We further visualize VCR’s cross-variable attention on ZafNoo in . In Figure 5, rows represent query variables, columns represent the variables they attend to, and colors indicate attention weights.
At both forecast horizons, most off-diagonal entries in the swc_deep column have relatively high weights, indicating that multiple variables draw on the historical representation of swc_deep when generating their correction vectors. The swc_shallow column is also more prominent at than at , illustrating how cross-variable attention weights differ between the two horizons. We will include this ablation and visualization in the final version of the paper and clarify the scope of the cross-variable dependencies that VCR models.
[1] TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods. Proceedings of the VLDB Endowment, 17(9), pp. 2363–2377, 2024.
Response to Reviewer J9p9 (Part I)
We thank Reviewer J9p9 for the careful review and specific suggestions. In response, we conducted additional experiments and analyses.
W1.1 & Q1: Lack of direct evidence for the “structural semantic gap” in the frozen MAE’s modeling of the temporal and phase axes and for TPR’s ability to bridge it.
We thank the reviewer for helping us strengthen the paper by asking us to clarify the structural semantic gap and provide further evidence for the role of TPR. The structural semantic gap refers to the different meanings of the two axes in natural images and time-series images. During natural-image pretraining, MAE learns spatial relationships among image patches. In a time-series grid formed by folding the sequence into periods, the horizontal axis tracks the same phase across successive historical periods, while the vertical axis indexes different phases within a period. After resizing and patch embedding, the historical time and phase axes retain these meanings at a coarser resolution. TPR models temporal and phase dependencies separately along the two axes.
To test whether this explicit modeling improves forecasting, we compare MUSE with only the TPR branch against VisionTS. Table 2 (lines 397–407 of the original paper) already reports the results for MUSE with TPR alone on ETTh2, Weather, AQWan, and ZafNoo at a forecast horizon of H=720. The table below reproduces those results alongside the VisionTS results for the same datasets and forecast horizon.
Comparison of MUSE with only the TPR branch and VisionTS. Each cell reports MSE / MAE. Lower is better, and the better result for each metric is in bold.
As shown in the table, MUSE with TPR alone achieves lower MSE and MAE than VisionTS on ETTh2, Weather, and ZafNoo, with the largest improvements on Weather. Across the four datasets, mean MSE decreases from 0.548 to 0.540 and mean MAE from 0.449 to 0.440. This comparison provides evidence at the forecasting level that explicitly modeling temporal and phase dependencies along the two axes helps adapt the frozen MAE’s visual representations to time-series forecasting. We will include this comparison and analysis in the final version of the paper.