- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates how to incorporate historical text into numerical time-series forecasting without degrading the temporal model. It compares six naive fusion strategies with four constrained strategies and proposes CFA, a residual adapter consisting of down-projection, LayerNorm, ReLU, and up-projection. The text encoder is frozen, while the time-series model and fusion parameters are trained. The primary study spans nine Time-MMD datasets, 14 forecasting backbones, four text encoders, and four horizons, giving 2,016 configurations. Additional analyses include synthetic matching/contradictory/irrelevant text, rank sensitivity, representation diagnostics, multiple seeds, and ETT experiments using text derived from historical numerical observations. CFA has the best aggregate normalized MSE and average rank among the reported fusion strategies, with small module overhead. However, gains are heterogeneous, and the evidence does not yet isolate low rank as the mechanism responsible for semantic filtering or preservation of temporal information.
Critical strengths
- An extensive and useful empirical comparison of fusion choices. The 2,016-configuration study covers several backbones, text representations, datasets, and horizons, with detailed per-configuration results. The supplied recount finds that CFA beats the no-text baseline in 1,204 configurations, ties in 127, and loses in 685. This breadth supports a meaningful aggregate-performance observation while exposing its limits. The paper also includes multi-seed analysis and an additional ETT benchmark, so the concern is not a complete absence of statistical or external evaluation.
- A simple, lightweight design with relevant robustness probes. The adapter is easy to integrate and has reported module overheads of 0.61% in parameters and 0.04% in FLOPs, although these exclude the full cost of text encoding. The synthetic matching, contradictory, and irrelevant-text tests address meaningful failure modes. A carefully substantiated empirical finding about such a simple component would be worthwhile even without claiming an entirely new adapter architecture.
Critical weaknesses
- The central explanation of text-dependent low-rank filtering is not established, and the stated computation may remove text dependence for some backbones. Appendix P uses a bottleneck width of
max(1, floor(C/r))for several channel-based backbones. With the reported channel counts and defaultr=8, these bottlenecks have width one. If Equation (1) applies standard LayerNorm over that single feature, its normalized value is constant: it equals the learned bias with affine normalization and zero without it. The resulting residual would then be independent of the input text. This is a conditional mathematical implication of the description, not proof of what the code does; a different normalization axis or special handling could resolve it. Independently, the “without bottleneck” comparison also changes normalization, nonlinearity, and parameterization, so it cannot isolate rank. Low rank constrains the injection subspace but does not itself bound residual magnitude or prevent cancellation of temporal information. These issues directly affect the claimed mechanism behind the contribution. See Equation (1) and Appendices I, J, K, L, and P. - The evidence for robust superiority and semantic selectivity is weaker than the aggregate claims imply. The Friedman/Nemenyi analysis treats 2,016 configurations as blocks despite shared datasets, horizons, backbones, and repeated no-text baselines. This dependence matters for the relatively small CFA–Gating rank difference of 0.361, only slightly above the reported 0.05 critical difference of 0.336. CFA beats Gating in 1,049 configurations but loses in 871, with 96 ties; superiority is not uniform. The semantic-injection diagnostic also reports a matching-versus-contradicting contribution-ratio difference of only about
1.94 × 10^-6on the original scale. A significant standardized effect does not by itself establish practically meaningful suppression, and text-template differences can explain the effect without checking consistency against the same history. Finally, extreme divergence in some naive-fusion configurations makes matched normalization, initialization, and convergence checks important to interpreting the average advantage. These concerns limit both the statistical and mechanistic conclusions, rather than negating the observed aggregate gains.
Questions for authors
- Decisive implementation clarification: For DLinear, FiLM, Koopa, and TSMixer, what are the actual tensor shapes, bottleneck widths, and LayerNorm axes? With numerical history fixed, does replacing the text change the adapter output when the bottleneck width is one? Please reconcile the dimension rules in Appendices I and P with the implementation.
- Assessment-changing ablation: When LayerNorm, ReLU, initialization, and training protocol are held fixed, how does low rank compare with full rank? How competitive are constant, zero, or shuffled-text residual controls? These comparisons would determine whether the gain genuinely depends on informative text and the rank constraint.
- Assessment-changing statistical check: Does CFA retain an advantage over Gating under dataset-level paired analysis or a suitable clustered/hierarchical analysis? Please provide per-dataset effects and intervals, and reconcile the different CFA–Gating Wilcoxon values in Tables 19 and 20.
- Assessment-changing mechanism check: With the same numerical history and matched text templates, does changing direction, timing, or factual content produce meaningful changes in residual norms, predictions, and errors? Please show the contribution ratios on their original scale and explain the practical significance of the observed difference.
- Clarification of the comparison protocol: How are hyperparameters selected on validation data, how is normalized MSE computed and aggregated, and do the main conclusions persist after stabilizing naive fusion with matched normalization and initialization or separately reporting divergent runs?
2 comments
Dear Reviewer XVi8,
As we are approaching the end of the discussion period, we appreciate your time and consideration. If you have any remaining questions or comments, we would be happy to address them before the deadline.
Best regards,
The Authors
Please let me know if you have any further questions or concerns!
If your concerns have been sufficiently addressed, I would greatly appreciate your consideration of a score increase.