- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
C4TS combines a PatchTST-style temporal pathway with a VisionTS-style visual pathway: a pretrained MAE reconstructs a masked, period-structured image, with only LayerNorm fine-tuned. The two pathways interact at two points. (i) Co-encoding: Spatial-Aware Visual Adaptation (Sec. 3.3, Eqs. 3–8) restores visible MAE tokens to their 2D grid, applies two dilated depthwise-conv residual blocks plus row/column embeddings, and uses the result as cross-attention memory for the temporal tokens (visual→temporal only). (ii) Co-decoding: Instance × Future-Patch Adaptive Fusion (Sec. 3.4, Eqs. 9–13) uses K learnable future-patch queries. These read stop-gradient temporal and visual memories and produce a sigmoid weight α_k per instance and future patch, broadcast across variables, to mix Ŷ_T and Ŷ_V. Training supervises both pathways and the fused output (Eq. 14). With L = 512 on 7 TSLib benchmarks, C4TS has the best average rank (2.071) and the lowest overall average MSE (0.298) among 9 baselines (Table 1). ETT ablations are in Table 2, and attention and gate visualizations in Fig. 3.
Critical strengths
- [Importance of the research problem] How to combine numerical forecasting with MAE-based visual reconstruction (VisionTS) is a timely question, and Fig. 1 positions it clearly against feature-level fusion (Time-VLM) and global prediction fusion (DMMV).
- [Novelty of the approach] Two well-motivated modules: Spatial-Aware Visual Adaptation, which turns the 2D MAE token grid into a cross-attention memory (Sec. 3.3), and Instance × Future-Patch Adaptive Fusion with learnable future-patch queries (Sec. 3.4, Eqs. 9–13).
- [Soundness and feasibility] The engineering is careful. The gate is zero-initialized so α starts at 0.5, gate inputs use stop-gradient, only visible historical tokens enter the adapter, and only the MAE LayerNorm parameters are fine-tuned.
- [Experiments and practical impact] All methods use a common L = 512, with reruns of DMMV, VisionTS, VPNet and FreqCycle at that length (Sec. 4.1). C4TS has the best average rank (2.071) and the lowest overall average MSE (0.298, Table 1).
- [Experiments and practical impact] The ablations are informative about direction: adding temporal→visual attention hurts (+2.747%, Table 2).
- [Theoretical and application-level implications] The results indicate that temporal and visual-reconstruction forecasters have complementary inductive biases. This is a useful insight for ensemble design.
- [Theoretical and application-level implications] Per-patch weights could serve as a horizon-level monitoring signal.
Critical weaknesses
- [Importance of the research problem] It is a fairly narrow architectural question within a fast-moving niche rather than a broadly important problem.
- [Novelty of the approach] Each module is an incremental variant of known ideas (cross-modal attention; input-dependent gating). The main step beyond DMMV is replacing a global scalar weight with an input- and horizon-dependent gate.
- [Soundness and feasibility] The fusion weight α_k is averaged over variables (Eq. 12) and broadcast across channels, even though channels in Weather, Electricity and Traffic differ greatly in periodicity.
- [Soundness and feasibility] There are no parameter counts, runtime or memory figures, despite running a ViT-scale MAE per variable.
- [Soundness and feasibility] Code is not available at submission.
- [Experiments and practical impact] Margins are small (0.298 vs. 0.302 for APRNet and 0.304 for VisionTS and PatchTST), with no seeds or standard deviations.
- [Experiments and practical impact] The module ablations are each under 1% (no spatial adaptation +0.603%, no feature interaction +0.302%, uniform averaging +0.748%), and they cover ETT only.
- [Experiments and practical impact] Most of the gain comes from averaging the two pathways (0.3481 → 0.3346 with uniform averaging).
- [Experiments and practical impact] Some baseline reproductions look anomalous (DMMV ETTh2 MSE 0.574, average rank 9.429).
- [Theoretical and application-level implications] There is no account of when the visual pathway should dominate. Fig. 3 shows that the weights vary, not that they track relative error.
- [Theoretical and application-level implications] Deployment cost per variable is high relative to an overall MSE gain of about 0.004, and the image construction assumes a known period.
Questions for authors
- Can you provide mean ± std over multiple seeds for Table 1 and Table 2, and paired tests for the co-decoding ablations?
- What is the MSE of an ensemble of a separately trained PatchTST (L = 512) and VisionTS, averaged uniformly or with a global weight?
- Why does DMMV degrade so much at L = 512 (e.g., ETTh2)? Did you try its native look-back?
- Did you try per-variable α_{c,k}? How do results change on high-dimensional datasets (ECL, Traffic)?
- Is there a correlation between the gate's α_k and the relative per-patch errors of the two pathways on held-out data?
- What are the parameter count and wall-clock cost compared with PatchTST and VisionTS?
8 comments
Responses to Critical Weaknesses for Reviewer WB7m - Critical Weakness 2
W2. Novelty of the approach and differences from DMMV
The architectural contribution of C4TS is to coordinate numerical forecasting and visual reconstruction at both the representation and prediction levels while preserving two complete forecasting pathways. This is realized through two complementary designs:
• Co-encoding: Spatial-Aware Visual Adaptation restores and processes the two-dimensional grid of historical MAE tokens, constructing a visual memory that enhances temporal representations through cross-attention. The original MAE encoding remains available to the visual decoder, preserving its reconstruction-based forecasting mechanism. • Co-decoding: Learnable future-patch queries retrieve information from the temporal and visual historical representations and generate fusion weights for each input instance and future patch. These weights directly combine the two completed forecasts over the prediction horizon. • Cross-attention and gating are the underlying operations. The contribution concerns how historical visual information is adapted for interaction, how both forecasting mechanisms are retained, and how their outputs are coordinated across future prediction positions. Sections 3.2–3.4 describe these architectural choices.
Compared with DMMV, the differences concern both co-encoding and co-decoding.
First, DMMV does not include the co-encoding mechanism used in C4TS. DMMV connects its forecasting pathways through seasonal–trend decomposition or a visual-backcast residual formulation and combines their outputs through prediction-level fusion. It does not construct a spatially adapted visual memory that temporal tokens query through cross-attention.
C4TS explicitly introduces this feature-interaction pathway. Spatial residual blocks and row–column positional embeddings adapt the historical visual token grid before temporal tokens read it. Meanwhile, the original MAE encoding continues to support visual reconstruction. This allows visual information to enhance temporal forecasting without replacing the visual forecasting pathway.
Second, the fusion mechanisms differ in the dimensions along which their weights can vary. To make this distinction precise, we use the notation [I, F, V, B]. The first three dimensions indicate whether weights vary across input instances, forecast positions or patches, and variables, respectively. A value of 1 indicates sharing along that axis. B describes the branch-weight parameterization: one coefficient determining complementary weights, or two separately learned coefficients.
The comparison below also includes an additional instance × variable × future-patch three-dimensional gate, which we evaluate as an ablation and discuss in detail in W3.
| Fusion mechanism | Representation | Weight sharing |
|---|---|---|
| Global single-parameter fusion | [1, 1, 1, 1] | Shared across instances, positions, and variables |
| Released DMMV-S | [1, 1, V, 1] | Variable-specific; shared across instances and forecast positions |
| Released DMMV-A | [1, 1, V, 2] | Two coefficients per variable; shared across instances and forecast positions |
| C4TS | [I, F, 1, 1] | Instance- and future-patch-dependent; shared across variables |
| Additional three-dimensional gate | [I, F, V, 1] | Instance-, future-patch-, and variable-dependent |
Throughout our manuscript, DMMV refers to DMMV-A. DMMV’s fusion weights are learned during training and shared across input instances and forecast positions at inference.
The distinction is that C4TS generates fusion weights conditioned on the current input and future patch, allowing the contributions of the two pathways to vary across instances and future prediction positions.
Specifically, the future-patch queries in Section 3.4 read the temporal and visual historical representations to generate a visual weight for each instance–future-patch pair. That weight directly combines the numerical forecast and visual reconstruction for the corresponding future patch.
Thus, the advance beyond DMMV is not limited to replacing its fusion coefficients. C4TS introduces spatially adapted feature interaction together with future-patch-conditioned prediction fusion, enabling collaboration during encoding while retaining and adaptively combining both complete forecasting pathways during decoding.
Responses to Critical Weaknesses for Reviewer WB7m - Critical Weakness 3
W3. Soundness and feasibility
a) Shared fusion weights across variables
Within each dataset, all variables are recorded at the same sampling interval. Consequently, common calendar cycles, such as daily cycles, correspond to the same number of time steps across channels. This provides a consistent temporal basis for constructing period-structured images and defining future patches.
The shared gate in C4TS does not require identical temporal characteristics across variables. Each variable retains its own temporal forecast and visual-reconstruction forecast; what is shared is the relative weighting of these two forecasting mechanisms within an instance–future-patch pair. The gate aggregates information across variables to determine this weighting, and its output changes with the input instance and future patch.
To directly evaluate whether variable-specific weights improve this design, we added two fusion controls in Section 4.3 and Appendix C, Table 5. Variable-wise fusion learns a separate coefficient for each variable, shared across instances and future patches. The instance × variable × future-patch three-dimensional gate, also listed in the comparison in W2, generates a separate coefficient for every variable within each instance–future-patch pair.
Across the four ETT datasets, four forecasting horizons, and four random seeds, variable-wise fusion achieves an average MSE of 0.3339, and the three-dimensional gate achieves 0.3328, compared with 0.3324 for the adopted instance × future-patch gate. Thus, explicitly allowing variable-specific fusion does not improve the overall ETT result. These controlled comparisons support sharing the fusion weights across variables while adapting them to the current input and future prediction position.
Given the substantially larger scale of Weather, Electricity, and Traffic, repeating the full ablation suite across multiple random seeds on these datasets would incur prohibitively high computational costs. We therefore conduct the systematic fusion-granularity comparisons on ETT and evaluate overall forecasting performance on all seven datasets.
b) Parameter counts, inference latency, and memory consumption
We have added parameter counts and measured inference costs in Appendix D, Table 7. The comparison includes C4TS, its temporal and visual branches, VisionTS, DMMV, and Time-VLM. For example, the ETTh1 results are:
| Model | Total parameters | Trainable parameters | Median latency | Peak memory |
|---|---|---|---|---|
| C4TS | 118.56M | 6.71M | 26.41 ms | 510.14 MiB |
| C4TS temporal branch | 6.17M | 6.17M | 3.12 ms | 35.13 MiB |
| C4TS visual branch | 111.91M | 0.06M | 19.00 ms | 485.39 MiB |
| VisionTS | 111.91M | 0.06M | 23.47 ms | 485.49 MiB |
| DMMV | 114.49M | 2.59M | 43.28 ms | 511.09 MiB |
| Time-VLM | 131.18M | 19.59M | 21.82 ms | 532.50 MiB |
Each entry reports the maximum of that metric across the four forecasting horizons. Measurements use the same GPU, an input length of 512, and a test batch size of 1. For each horizon, latency is the median of 50 forward passes following 10 warm-up passes. Appendix D documents the execution settings.
The C4TS temporal branch serves as the temporal-only reference. It is a separate implementation inspired by PatchTST-style modeling, rather than the official PatchTST implementation. Holding this temporal architecture fixed directly quantifies the additional cost of introducing the visual pathway and collaboration modules.
The MAE parameters are shared across variables, rather than replicated into a separate parameter set for each channel. Processing more variables nevertheless increases computation and activation memory. Most MAE parameters remain frozen, which explains the difference between total and trainable parameter counts. The measurements make these computational costs explicit alongside the forecasting results.
c) Code availability and reproducibility
The supplementary c4ts_minimum package provides a minimal C4TS evaluation setup for verification during the review process. It includes model inference code, data-loading utilities, environment specifications, and instructions for obtaining the checkpoint and dataset. The package supports evaluation on ETTh1 with a forecast horizon of 96 and provides reference metrics and checksum verification.
We also provide the dmmv_logs directory, containing all run settings, execution logs, and recorded results from our DMMV reproduction experiments.
As stated in the manuscript’s reproducibility statement, we will release the complete code, training and evaluation scripts, and experimental configurations upon acceptance.
Responses to Critical Weaknesses for Reviewer WB7m - Critical Weakness 1
Dear Reviewer WB7m,
Thank you for your detailed assessment and constructive suggestions. We have expanded the evaluation with four-seed ablations, additional fusion-granularity controls, independent-pathway ensembles, a random-initialization control for the MAE, and parameter and inference-cost measurements. We also provide DMMV reproduction logs and a minimal C4TS evaluation package in the supplementary material.
Below, we respond point by point to your comments. W1–W5 correspond to the five thematic categories under “Critical Weaknesses,” in their original order. Within each category, we address its individual comments separately. Q1–Q6 correspond to the six items under “Questions For Authors,” in their original order. Section and table numbers refer to the revised manuscript.
W1. Importance of the research problem
Time series forecasting supports decision-making in a wide range of applications, including energy management, transportation, and weather monitoring. Improving the ability to predict future behavior from historical observations therefore has practical value across multiple domains.
Architectural advances remain an important way to improve forecasting. Different architectures encode different assumptions about temporal structure and determine which patterns can be extracted and used for prediction. In particular, numerical forecasting and visual reconstruction provide complementary ways to process the same observations: one directly models numerical sequences, while the other exposes temporal patterns through a structured image representation. Understanding how these forecasting mechanisms can work together is therefore a meaningful architectural question.
C4TS investigates this question beyond simply adding visual features to a temporal predictor or averaging two forecasts. Its central objective is to enable collaboration between numerical forecasting and visual reconstruction while preserving both as complete forecasting pathways. This raises two concrete design questions:
• Co-encoding: How can historical visual representations be adapted to enhance temporal forecasting through feature interaction while preserving the MAE’s visual reconstruction mechanism?
• Co-decoding: How can the two completed forecasts be combined adaptively across input instances and future prediction positions?
Sections 3.2–3.4 address these questions through spatially adapted co-encoding and instance × future-patch co-decoding. The revised experiments further distinguish the contributions of individual pathways, independent ensembles, feature interaction, and adaptive prediction fusion. In particular, the independent uniform and validation-weighted ensembles provide direct controls for determining whether the collaborative framework offers benefits beyond combining separately trained forecasters.
The value of this study therefore lies in both the resulting forecasting model and the experimental examination of how two different forecasting mechanisms can collaborate at the representation and prediction levels. The seven-dataset evaluation assesses forecasting performance, while the expanded controlled ablations examine the architectural choices underlying that collaboration.