acceptodds
Under review as a conference paper at ICLR 2027

TIFUSE: SELECTIVE AND STABLE TIME–FREQUENCY FUSION FOR CLOUD WORKLOAD FORECASTING

Abstract

Cloud workloads combine recurring periodicities with abrupt shifts and transient disturbances, requiring models to capture global structures and local dynamics simultaneously. Existing time–frequency forecasters typically learn both domains exhaustively and fuse them directly, which can preserve redundant information and make cross-domain interactions less stable. We propose TiFuse, a selective and stable time–frequency framework for cloud workload forecasting. TiFuse constructs a compact spectral representation by retaining energy-dominant DCT components, and a complementary temporal representation by selecting representative multi-scale historical patches while preserving recent dynamics. Their interaction is modeled with relation-enhanced attention that couples sample-specific dependencies with a global relational prior under input-dependent sparse constraints. Experiments on five real-world cloud traces covering CPU, memory, and GPU workloads show that TiFuse achieves the lowest average MSE in 15 of 16 dataset–resource settings and the lowest average MAE in 13 of 16, while maintaining competitive computational efficiency.

Primary area:
learning on time series and dynamical systems
Keywords:
Cloud Workload Forecasting, Multivariate Time Series, Adaptive Frequency Selection, Time–Frequency Fusion

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7qs3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TiFuse for cloud workload forecasting. It combines a frequency branch that selects energy-dominant DCT components with a temporal branch that selects historical patches from a multiscale representation while retaining recent patches. The resulting tokens are processed by relation-enhanced attention, which adds a learned global relational prior to sample-dependent attention under a binary mask. Experiments cover CPU, memory, and GPU forecasting on five cloud traces, with results averaged over four forecasting horizons. TiFuse reports the lowest average MSE in 15 of 16 dataset–resource settings and competitive training time and memory consumption.

Critical strengths
  1. The method addresses a practical forecasting problem and makes a sensible distinction between recurring workload patterns and recent dynamics. In particular, retaining recent patches alongside selected historical patterns reduces the risk of losing information near the forecasting boundary. The DCT branch also provides a compact representation without requiring an inverse transform.

  2. The experiments cover several real cloud traces and different resource types. TiFuse performs well against both general forecasting methods and a workload-specific baseline, FP-MLP. The training-time and memory comparisons are also useful. For example, Table 5 shows memory consumption close to PatchTST and PatchMixer, although these models are generally faster. Together, these results suggest a useful balance between forecasting accuracy and computational cost.

Critical weaknesses
  1. The supplied implementation does not fully match the stated method and experimental protocol. In DAFC.py, spectral energy is averaged over the batch, variables, and within-patch positions, producing one shared set of frequency indices. This differs from the input-specific selection described in Eqs. (4)–(6), and means that a sample’s representation can change depending on the other samples in its inference batch. The default selection also forces the DC component into the frequency budget even when it is outside the energy top-KK, so the unconstrained maximum-energy property in Appendix A does not directly apply to this implementation. Separately, Dataset_Custom assigns 70% of the data to training and 20% to testing, leaving approximately 10% for validation, whereas Appendix B specifies 7:2:1 for training, validation, and testing. These differences need to be reconciled with the configurations used to produce the reported results.

  2. The central claim of stable fusion is not directly demonstrated. The attention visualizations in the main paper and Appendix C show increased matrix rank and different connection patterns, but neither establishes robustness to noise or workload shifts. Adding a learned matrix can itself increase rank. The Ori-Attn ablation supports a forecasting benefit, but removes the prior and mask together, leaving their individual roles unclear. The shared prior also operates over energy-rank and selection-channel positions whose underlying frequencies and historical locations vary across samples. This design may work, but the current evidence does not explain why it provides stable cross-domain relationships. The prediction plots are useful illustrations, though they do not provide a controlled comparison of this claim.

Questions for authors
  1. Was DAFC evaluated using per-sample frequency selection or the batch-and-variable-averaged selection in the released code? If the latter, how sensitive are predictions to inference batch size and batch composition? Was the DC component always forced into the selected set?

  2. Which split was used for the reported results: the 7:2:1 split described in Appendix B or the approximately 7:1:2 split implemented in Dataset_Custom? Please provide the exact configurations or commands used for the main experiments.

  3. Appendix B states that results are reported as means and sample standard deviations over five runs, with ten runs for the Usage CPU comparison. However, Tables 6–8 show only point estimates. Please provide the standard deviations, especially for the close comparisons with FP-MLP.

  4. Can the authors separate the effects of the global prior and the sparse mask, and provide a direct comparison under workload shifts or input perturbations? How does the prior remain useful when a token position corresponds to different frequencies or historical patches across samples?

4 comments

1

Thank you for pointing out this discrepancy. The manuscript describes frequency selection for an individual input. The code selects frequencies using energy aggregated across the batch. To simplify the notation and make the computation easier to follow, we first describe DAFC using a single-channel sequence. PatchTST also uses a univariate sequence to illustrate the computation of a multivariate model. Section 3.2 states, “To illustrate the computation of DAFC, we first consider a single-channel historical sequence.” In this setting, the energy definition and frequency selection in Eqs. (4)–(6) are mathematically valid.

In the public implementation, we also aggregate energy across the batch and variable dimensions, extending Eq. (4) as follows.

Eˉq=1BCp∑b=1B∑c=1C∑u=1p∣Zb,c,u,q∣2\bar{E}_q=\frac{1}{BCp}\sum_{b=1}^{B}\sum_{c=1}^{C}\sum_{u=1}^{p}\left\lvert Z_{b,c,u,q}\right\rvert^{2}

We use the aggregated energy to select the frequency indices shared across the entire batch. Here, BB, CC, and pp denote the batch size, the number of variables, and the patch length, respectively. When B=C=1B=C=1, this definition reduces to Eq. (4). The original manuscript did not fully explain the batch aggregation or the use of shared indices, leaving the distinction between per-sample and batch-level selection unclear. We have clarified this distinction in Appendix A.1.

During model development, we used the experimental option keep_dc=True to compare strategies for handling the DC component. This option was specific to those development experiments. We have since refined and committed the code.

To assess the sensitivity of TiFuse to inference batch size and composition, we ran additional memory workload forecasting experiments at a forecast horizon of H=5H=5. Table 11 uses a batch size of 512 as the reference and compares it with batch sizes of 1, 64, and 128. Table 12 compares the original batch grouping with two alternative groupings generated using random seeds 2026 and 2025.

Table 11 (Appendix D). Memory workload forecasting errors at different inference batch sizes (H=5H=5)

Batch sizeUsage MSEUsage MAERequest MSERequest MAEAlibaba MSEAlibaba MAEGoogle2019 MSEGoogle2019 MAE
10.00150.02420.00590.04720.28000.31500.03820.1340
640.00130.02360.00550.04680.27670.30960.03790.1317
1280.00140.02350.00570.04700.27810.31760.03700.1316
5120.00140.02400.00570.04630.27370.31330.03740.1335

Table 12 (Appendix D). Memory workload forecasting errors under different inference batch groupings (H=5H=5)

Batch groupingUsage MSEUsage MAERequest MSERequest MAEAlibaba MSEAlibaba MAEGoogle2019 MSEGoogle2019 MAE
Random seed 20260.00130.02400.00580.04570.26840.30910.03790.1311
Random seed 20250.00150.02410.00590.04760.27900.30950.03680.1350
Original grouping0.00140.02400.00570.04630.27370.31330.03740.1335

Under these test conditions, test-set MSE and MAE remained broadly stable across inference batch sizes and groupings.

2

The original manuscript incorrectly reported a data split ratio of 7:2:1. Both the code and the experiments used 7:1:2 throughout. We have corrected the typographical error in the revised manuscript and updated the detailed experiment configuration in the code.

3

We thank the reviewer for pointing out this omission. Tables 6–8 in the revised manuscript now report means and sample standard deviations over five runs. The Usage-CPU comparison is based on ten runs.

4

The global prior supplements the estimates of interaction weights derived from the current window alone. It acts as a memory bank learned during training, storing interaction preferences between token positions that are shared across samples. The input-dependent mask determines where the prior is applied. When a connection has a high enough matching score to enter the candidate set for the current window, the model adds the corresponding prior to the base attention score. The current window thus determines which time–frequency connections receive a prior adjustment, and the learned prior modulates their weights during information aggregation.

The global prior encodes learned interaction preferences for pairs of token positions, even when the same positions represent different frequencies or historical segments across samples. For example, the same pair of positions may represent two low-frequency components in sample A and two different historical segments in sample B. The pair shares the same prior value across both samples, while the model recomputes its matching score from the token representations in each window. If the pair enters the candidate set only in sample A, the prior adjusts the attention weight for that connection only in sample A. If the pair enters both candidate sets, the prior adds the same bias in both samples. The final weights can still differ because the base attention scores can differ. The model can therefore apply learned interaction preferences to connections supported by the current input even when the time–frequency content at a given position varies across samples.

To isolate the effects of the components, we added a control that retains the global prior with its input-dependent mask removed. Using the same original inputs, we compared forecasting errors and their increases across the three model variants.

DatasetHorizon HStandard attention MSEStandard attention MAEUnmasked prior MSEUnmasked prior MAEFull TiFuse MSEFull TiFuse MAE
Alibaba10.093090.171960.092710.166990.090520.16157
Alibaba50.274420.318240.273250.315410.273700.31328
Alibaba100.416450.420290.409920.412840.401590.40563
Alibaba300.618550.513990.608640.510870.592130.50775
Google201910.008190.063880.007990.060810.007000.05774
Google201950.039680.138910.038590.136220.037430.13352
Google2019100.078750.199440.077390.196640.073840.19195
Google2019300.209460.341040.208590.339910.207420.33692

Thank you for clarifying the batch-level frequency selection and the 7:1:2 data split, and for adding the standard deviations and batch sensitivity results. These additions address most of my questions about the experimental setup. The unmasked-prior comparison helps separate the effects of the prior and the mask on forecasting accuracy. My main concern remains the claim of stable fusion. The new ablation uses the original inputs, so it remains unclear whether the mask helps under input perturbations or workload shifts. Given this remaining concern, I am keeping my original rating.

Thank you for raising this concern. We evaluated three attention variants on the Alibaba and Google2019 memory workload datasets at all four forecast horizons. The variants were standard attention, attention with an unmasked relational prior, and Full TiFuse with the input-dependent mask. We added zero-mean Gaussian noise only to the historical observations in each test window. For each window, we applied the same noise realization to all three variants and kept the test windows, inference batch grouping, and forecast targets fixed.

Under Gaussian perturbation, Full TiFuse achieved lower MSE and MAE than the unmasked-prior variant in every evaluated setting. We measured error increases relative to each variant's performance on the original inputs. For both MSE and MAE, Full TiFuse showed a smaller increase in most settings. These results suggest that the input-dependent mask mitigates the degradation in forecasting accuracy under the tested Gaussian perturbation in most settings, with the effect varying across datasets and forecast horizons.

Memory workload forecasting errors after adding Gaussian noise to the historical inputs

DatasetHorizon HHStandard Attention MSEStandard Attention MAEUnmasked Prior MSEUnmasked Prior MAEFull TiFuse MSEFull TiFuse MAE
Alibaba10.114870.205710.113200.202750.110530.19726
Alibaba50.370540.397900.360870.393240.350980.38073
Alibaba100.557610.549430.541610.533390.537660.52758
Alibaba300.761120.682160.745670.666670.722030.65834
Google201910.009080.074100.008990.073170.008810.07090
Google201950.052030.178340.051550.174260.049830.17208
Google2019100.102450.254120.101080.246900.097620.23611
Google2019300.289520.463670.282770.452400.273610.44073
Reviewer_h2LEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors presnet a selective and stable time–frequency framework for cloud workload forecasting. The overall design of TiFuse is reasonably well-motivated. In particular, the idea of introducing selective representation learning into time–frequency fusion, rather than directly modeling all available information, is a promising direction.

Critical strengths

The overall design of TiFuse is reasonably well-motivated. In particular, the idea of introducing selective representation learning into time–frequency fusion, rather than directly modeling all available information, is a promising direction.

Critical weaknesses

none.

Questions for authors

1.The authors discuss the impact of redundant temporal information on interaction stability. However, does information redundancy necessarily lead to unstable interactions? 2.TiFuse constructs a compact spectral representation by retaining nergy-dominant DCT components. However, some low-energy frequency components may contain early indicators of sudden workload changes. Could the proposed selection strategy discard such informative low-energy components, potentially leading to forecasting failures? 3. The authors emphasize that the proposed model can stably capture eterogeneous time–frequency interactions. However, the architecture does not explicitly illustrate distinct interaction pathways between the temporal and frequency domains. It would be helpful to clarify how the proposed mechanism explicitly models cross-domain interactions rather than simply fusing temporal and frequency features.

3 comments

1

Thank you for pointing out the ambiguity in the original wording. We have revised the abstract and introduction to clarify what we mean by “redundancy” in the context of forecasting. The input may retain information that is only weakly relevant to the forecast or content affected by noise or anomalous perturbations. In TiFuse, attention mediates interactions between the time and frequency domains. Redundant content can make these interactions less stable. Kim et al. observed that standard dot-product attention can assign high weights to corrupted segments in real-world time series, resulting in less stable forecasts. CSP aims to minimize this instability by selecting representative segments from multiscale historical segments. The ablation in Table 4 supports the forecasting value of this selection. Removing CSP and retaining all time segments led to a mean MSE of 0.00603 across the four forecast horizons, compared with 0.00391 for the full TiFuse model.

Kim et al., Local Geometry Attention for Time Series Forecasting under Realistic Corruptions, ICLR 2026

2

DAFC is designed to retain as much of the input's dominant spectral structure as possible with a small budget for frequency components. This design keeps computation efficient while maintaining good forecasting accuracy, as shown by the efficiency analysis and the main experiments. TiFuse complements its compact spectral representation with recent time-domain segments that bypass DCT component truncation and with representative historical segments. When DAFC omits some informative low-energy components, the time-domain branch can still convey related recent local variations to the forecaster. TiFuse can therefore still make useful forecasts when it omits a small amount of informative content, with substantial gains in computational efficiency. To our knowledge, accurately identifying all information relevant to a forecast in advance remains an open problem and a direction for our ongoing work.

3

Thank you for the suggestion. We have revised Section 3.4 to clarify how joint self-attention explicitly models cross-domain interactions. We concatenate the time-domain and frequency-domain tokens and feed them into the same self-attention module. The rows of the attention matrix correspond to queries, and the columns correspond to keys. The two off-diagonal blocks capture information flow in opposite directions.

For the frequency-to-time pathway, time-domain queries are scored against frequency-domain keys. These scores determine how frequency-domain value vectors are aggregated into the time-domain workload representations. For the time-to-frequency pathway, frequency-domain queries are scored against time-domain keys. The resulting scores govern the aggregation of time-domain value vectors from recent and selected historical segments, allowing frequency-domain representations to incorporate the current workload state and relevant historical patterns. Through these two blocks, joint self-attention aggregates information from each domain directly into the representations of the other.

We have also clarified in Section 3.4 how the global prior modulates these interactions. The global prior supplements the estimates of interaction weights derived from the current window alone. It acts as a memory bank learned during training, storing interaction preferences between token positions that are shared across samples. The input-dependent mask determines where the prior is applied. When a connection has a high enough matching score to enter the candidate set for the current window, the model adds the corresponding prior to the base attention score. The current window thus determines which time–frequency connections receive a prior adjustment, and the learned prior modulates their weights during information aggregation.

My concerns have been well addressed. Therefore, I decide to raise my score to 3 and recommend weak acceptance of this paper.

Thank you very much for your thoughtful review and for taking the time to reconsider our manuscript in light of our response. We are glad that our response addressed your concerns, and we sincerely appreciate your constructive feedback and the opportunity it gave us to clarify our work.

Reviewer_s9ZjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

TiFuse addresses cloud workload forecasting through compact time–frequency representations. It combines energy-based DCT component selection, learned selection of multiscale historical patches with a recent-history bypass, and attention augmented by a masked learned relational prior. Experiments cover five cloud traces, 16 dataset–resource settings, a 720-step lookback, and horizons of 1, 5, 10, and 30 steps. The reported horizon-averaged results place TiFuse best or tied-best in MSE for 15 of 16 settings, including one displayed tie, and best in MAE for 13 of 16. The main promise is the coordinated selection and fusion design. However, discrepancies between the formulation, supplied code, and stated protocol currently prevent a reliable interpretation of those gains.

Critical strengths
  1. Relevant problem and broad evaluation. The study covers CPU, memory, and GPU workloads across five traces, compares against eight baselines, and reports gains across many settings. This breadth makes the proposed direction practically relevant and provides more evidence than a narrow single-trace demonstration.
  2. Coherent architectural design. The spectral compression, selective historical representation, and recent-history bypass form a plausible complementary design. The transform, selection, and attention ablations provide useful initial evidence. The energy-retention argument is mathematically valid for the stated orthonormal transform of the patch matrix, although it does not establish predictive optimality or robustness.
Critical weaknesses
  1. The supplied spectral selector is batch-dependent rather than per-input, with a potential non-causal evaluation path. Equations (4)–(6), lines 230–248, describe selecting components from an input's energy. In layers/DAFC.py, lines 49–52, energy is instead averaged over the batch, variables, and patch offsets, producing one frequency-index set for every sample in the batch, including at inference. With chronologically ordered overlapping test windows, later samples can contain observations unavailable at earlier forecast origins. An earlier prediction can therefore depend on those observations through the shared index selection. A small synthetic check confirms that changing a later-window value can change the indices applied to an unchanged earliest input; it does not measure trained-model score inflation. In addition, keep_dc=True forces DC inclusion, while keep_dc=False excludes DC; neither option is necessarily the unconstrained all-bin top-K operation in the corollary. Without the exact implementation that generated the tables and causally valid evaluation results, the central accuracy evidence is not secure.
  2. The evaluation protocol and uncertainty are insufficiently auditable. Appendix B, lines 96–102, specifies a 7:2:1 train/validation/test split, whereas data_provider/data_loader.py, lines 233–237, implements 7:1:2 in Dataset_Custom. The supplied materials do not map each reported experiment to an exact configuration and implementation. Appendix B, lines 120–124, promises five-seed means and sample standard deviations, plus a prespecified ten-seed Usage-CPU setting, but Tables 1–4 and 6–10 give point estimates without the corresponding dispersion. This matters for small margins such as Usage-CPU MAE 0.0709 versus 0.0711 and the displayed Usage-Memory MSE tie at 0.0039. The module-level ablations and attention-rank analysis also do not isolate the prior's contribution or substantiate the central claim of stable learning.
Questions for authors
  1. Which exact code version produced the tables? Please reconcile the per-input formulation with batch-aggregated selection and the DC constraint, and provide results using per-sample causal selection or a train-only fixed selector, including a batch-composition invariance check.
  2. Which split and configurations were used for each reported experiment? Please reconcile 7:2:1 versus 7:1:2 and provide the promised seed-level results or means with sample standard deviations, including the ten-run Usage-CPU setting.
  3. Under the same corrected protocol, how does TiFuse compare with last-value persistence at each horizon, using identical target definitions, forecast origins, normalization, and metric aggregation? Given the short horizons, this simple reference would materially affect the assessment of the added forecasting value.
2 comments

1

Thank you for pointing out this discrepancy. The manuscript describes frequency selection for an individual input. The code selects frequencies using energy aggregated across the batch. To simplify the notation and make the computation easier to follow, we first describe DAFC using a single-channel sequence. PatchTST also uses a univariate sequence to illustrate the computation of a multivariate model. Section 3.2 states, “To illustrate the computation of DAFC, we first consider a single-channel historical sequence.” In this setting, the energy definition and frequency selection in Eqs. (4)–(6) are mathematically valid.

In the public implementation, we also aggregate energy across the batch and variable dimensions, extending Eq. (4) as follows.

Eˉq=1BCp∑b=1B∑c=1C∑u=1p∣Zb,c,u,q∣2\bar{E}_q=\frac{1}{BCp}\sum_{b=1}^{B}\sum_{c=1}^{C}\sum_{u=1}^{p}\left\lvert Z_{b,c,u,q}\right\rvert^{2}

We use the aggregated energy to select the frequency indices shared across the entire batch. Here, BB, CC, and pp denote the batch size, the number of variables, and the patch length, respectively. When B=C=1B=C=1, this definition reduces to Eq. (4). The original manuscript did not fully explain the batch aggregation or the use of shared indices, leaving the distinction between per-sample and batch-level selection unclear. We have clarified this distinction in Appendix A.1.

During model development, we used the experimental option keep_dc=True to compare strategies for handling the DC component. This option was specific to those development experiments. We have since refined and committed the code.

To assess the sensitivity of TiFuse to inference batch size and composition, we ran additional memory workload forecasting experiments at a forecast horizon of H=5H=5. Table 11 uses a batch size of 512 as the reference and compares it with batch sizes of 1, 64, and 128. Table 12 compares the original batch grouping with two alternative groupings generated using random seeds 2026 and 2025.

Table 11 (Appendix D). Memory workload forecasting errors at different inference batch sizes (H=5H=5)

Batch sizeUsage MSEUsage MAERequest MSERequest MAEAlibaba MSEAlibaba MAEGoogle2019 MSEGoogle2019 MAE
10.00150.02420.00590.04720.28000.31500.03820.1340
640.00130.02360.00550.04680.27670.30960.03790.1317
1280.00140.02350.00570.04700.27810.31760.03700.1316
5120.00140.02400.00570.04630.27370.31330.03740.1335

Table 12 (Appendix D). Memory workload forecasting errors under different inference batch groupings (H=5H=5)

Batch groupingUsage MSEUsage MAERequest MSERequest MAEAlibaba MSEAlibaba MAEGoogle2019 MSEGoogle2019 MAE
Random seed 20260.00130.02400.00580.04570.26840.30910.03790.1311
Random seed 20250.00150.02410.00590.04760.27900.30950.03680.1350
Original grouping0.00140.02400.00570.04630.27370.31330.03740.1335

Under these test conditions, test-set MSE and MAE remained broadly stable across inference batch sizes and groupings.

2

Thank you for identifying the typographical error in the data split ratio. The original manuscript incorrectly reported 7:2:1. Both the code and the experiments used 7:1:2. In the resubmitted manuscript, we have corrected the ratio and added the means and sample standard deviations of the experimental results to Tables 6–8. We have also updated the detailed experiment configuration in the code.

3

Thank you for suggesting a persistence baseline. We compared TiFuse with a persistence baseline on Alibaba memory request forecasting and Google2019 memory usage forecasting at horizons of H=1,5,10,30H=1,5,10,30. At each forecast origin, the persistence baseline predicts the last available observation of the target for each of the next HH steps. Both methods use identical target definitions, test forecast origins, and ground-truth target values. Standardization uses parameters estimated from the training set alone. To compute MSE and MAE, we average squared errors and absolute errors, respectively, on the standardized scale across all test forecast origins and forecast steps.

TiFuse achieved lower MSE and MAE than the persistence baseline at all four horizons on both datasets. These reductions were present even at the one-step horizon, indicating that TiFuse provides predictive value beyond carrying forward the last observation.

DatasetHTiFuse MSEPersistence MSEMSE improvementTiFuse MAEPersistence MAEMAE improvement
Alibaba10.09050.1163+22.185%0.16160.1895+14.727%
Alibaba50.27370.3675+25.526%0.31330.4044+22.527%
Alibaba100.40160.5864+31.513%0.40560.7041+42.393%
Alibaba300.59211.0278+42.393%0.50780.7749+34.469%
Google201910.00700.0082+14.958%0.05770.0697+17.257%
Google201950.03740.0489+23.587%0.13350.1829+26.995%
Google2019100.07380.1157+36.224%0.19200.2904+33.890%
Google2019300.20740.4448+53.374%0.33690.5760+41.510%

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.