Beyond Plug-and-Play: Do Forecasting Plugins Pay Off Across Models and Settings?

Forecasting plugins are increasingly proposed as plug-and-play or model-agnostic enhancements for time-series forecasting. However, architectural compatibility alone does not establish their effectiveness or practical deployment value across datasets, prediction horizons, and backbone models. We introduce PlugBench-TS, a controlled benchmark of ten forecasting enhancements spanning jointly fitted modules, training objectives, normalization, retrieval, output adaptation, and online forecast revision. It measures paired forecasting gain, checkpoint-pairing dependence where replacement is valid, and deployment memory and latency. The primary matrix covers 960 plugin–context combinations across four backbones, six datasets, and four prediction horizons, each evaluated in three paired runs. The results show why compatibility and a single accuracy number are not enough: the same enhancement can help in one context and hurt in another; a gain with its fitted backbone may disappear after a checkpoint change; and a small parameter count can conceal substantial retrieval memory and inference time. Building on these observations, we study whether the state used by online, static, and retrieval plugins can be reduced while preserving useful forecasting gains. The case studies reveal both successful reductions and accuracy–cost trade-offs. Overall, PlugBench-TS provides a reproducible basis for evaluating plugin value and for guiding future plugin design and deployment.