LG-Transformer: LLM-Guided Discovery of Boosted Novel Transformer Models for Time-Series Modeling

Transformer-based models have achieved strong results in long-term time-series forecasting, yet their architectures are still largely designed manually, leaving a vast combinatorial design space underexplored. To address this issue, we present LG-Transformer, an LLM-guided neural architecture search framework that automatically discovers high-performing Transformer variants for time-series modeling. Our method combines a pretrained large language model controller, with a weight-sharing supernet to enable context-aware architecture generation and efficient candidate evaluation. The search space spans key encoder design choices, including network depth, attention configuration, normalization placement, activation function, feed-forward expansion ratio, and residual operators. After search, the selected architecture is retrained from scratch, yielding a standard forecasting model with no additional inference-time overhead. Experiments on widely used multivariate forecasting benchmarks, including ETT, Electricity, Traffic, Weather, and ILI, show that the architectures discovered by LG-Transformer consistently outperform strong manually designed baselines and achieve highly competitive results across diverse prediction settings. These results demonstrate that LLM-guided architecture search is an effective and practical route to better time-series Transformers, while lowering the barrier for small industrial practitioners and startups to adopt large-model-driven solutions.