acceptodds
Under review as a conference paper at ICLR 2027

EnergyCA-LLM: Context-Guided Window-Adaptive Residual Learning for Unified Renewable Generation and Load Forecasting

Abstract

Growing renewable penetration is broadening power-system forecasting from isolated load prediction to unified forecasting of electricity demand, wind power, photovoltaic generation, and net load. These tasks share temporal structure but differ in physical drivers, operating regimes, and forecast-origin information availability. Existing methods study them separately or use static, task-agnostic context, limiting dynamic specialization. We propose EnergyCA-LLM, the first LLM-guided framework to jointly forecast electricity demand, wind power, photovoltaic generation, and net load within a single shared numerical backbone through forecast-origin-aware, window-specific residual adaptation. Per window, an availability-aware schema organizes the forecast-origin information set, covering task semantics, asset and geographic attributes, calendar and weather information, recent operating states, and field availability. A frozen pretrained LLM encodes this schema into a shared semantic state that interacts with the numerical latent state to produce residual codes. These codes gate multilayer low-rank residual adapters for task- and regime-specific modulation of shared temporal representations. Across four public benchmarks, chronological evaluation with four complementary point-forecasting metrics shows strong and competitive results against task-specific energy, general numerical, and LLM-enhanced baselines. Unified-training, encoder-replacement, and adapter-control ablations assess the roles of parameter sharing, pretrained semantic encoding, and dynamic residual adaptation.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Large Language Models, Multitask Learning, Parameter-Efficient Adaptation, Context-Aware Learning, Representation Learning, Renewable Energy Forecasting, Conditional Adaptation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7URvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EnergyCA-LLM, one shared model that forecasts load, wind power, PV generation and net load, each at its native cadence and horizon. The targets are hourly load from GEFCom2014 Load Task 15, 10-minute wind power from SDWPF, hourly PV generation from MMSP/FusionSF, and half-hourly regional net load from the Great Britain GSP-group benchmark. For each window, a deterministic constructor turns the information available at the forecast origin into text. The schema covers task, asset, geography, issuance time, calendar, weather, recent operation and field availability. Continuous weather and operating variables stay in the numerical stream, and the text carries low-cardinality regime descriptors defined on the training split. A frozen Qwen3-8B context encoder reads this text, and masked pooling followed by a trainable projection gives a semantic state. The numerical backbone separates endogenous and exogenous inputs as in TimeXer and uses cross-attention to forecast-known covariates. The controller pools the unmodulated base state, embeds field availability, and combines them with the semantic state into a window code. This code sets the rank and strength gates of shared low-rank adapters in each selected layer. A shared decoder with task- and cadence-conditioned queries produces the forecasts. Training uses a task-balanced, scale-normalized Huber loss plus a residual penalty on the low-rank corrections. Under a common training budget of at most 2,560 optimizer updates and three seeds, the paper reports the lowest error in 15 of the 16 task-metric comparisons against twelve baselines, using NMAE, RMSE, sMAPE and MASE. The one exception is PV RMSE, where FusionSF reaches 0.099877 and EnergyCA-LLM 0.100042. Table 2 reports unified-training, encoder-replacement and adapter-control ablations, which compare independent and shared numerical models, several replacements for the context encoder, and a fixed shared residual adapter.

Critical strengths

The paper makes explicit what is known at the forecast origin. It defines an information set at issuance time, a field-level availability mask, four source-status tags named static, observed, forecast and known-future, and regime descriptors defined from training data, with descriptor thresholds fixed before validation and testing. These are set out in Section 3, Section 4.1 and Appendix A.2. One shared checkpoint per seed serves all four task families and keeps the native cadences, channel sets and horizons of each. Trajectory prediction stays in the numerical model, and the frozen LLM only encodes context to control adaptation. This is a reasonable answer to the caution the authors cite from Tan et al. (2024), that some gains in LLM-enhanced forecasters arise from architecture and training rather than language-specific capabilities. The setting matters for grid operation, and it is the main reason I see value in this work.

The comparison covers four public benchmarks and four metrics. It includes general numerical and LLM-enhanced baselines as well as task-specific references, namely Load-GAM, the SDWPF Future-SCADA predictor, Clear-Sky, FusionSF and a GAM for net load. Results use three seeds, and Table 2 has three ablation panels. Some margins are large. Load sMAPE falls from 5.16 to 3.11, Wind sMAPE from 96.81 to 35.09, and Wind NMAE from 0.31 to 0.25. The authors also report their loss on PV RMSE openly. If the evaluation protocol holds up, these results would be useful to practitioners.

Critical weaknesses

Appendix A.3 states that when each task uses one global normalization scale for each metric, fixed across methods and seeds, NMAE = A/U and MASE = A/S. Then MASE = c × NMAE with c = U/S, so the seed mean and the population standard deviation both scale by c. The Table 1 means fit a single c per task across all rows within rounding, with c at about 6.27 to 6.65 for Load, 0.770 to 0.782 for Wind, 2.455 to 2.483 for PV and 7.70 to 7.79 for Net load. The standard deviations shown for every baseline also fit this ratio within rounding. The EnergyCA-LLM row does not. Table 2 gives three-decimal NMAE values from the same checkpoints. On Load, 0.031 ± 0.009 implies a MASE standard deviation of about 0.053 to 0.063, but Table 1 reports 0.20 ± 0.01. On PV, 0.264 ± 0.009 implies about 0.021 to 0.024, but Table 1 reports 0.65 ± 0.09. On Net load, 0.063 ± 0.003 implies about 0.019 to 0.027, but Table 1 reports 0.50 ± 0.08. On Wind, 0.253 ± 0.026 implies about 0.020 to 0.021, but Table 1 reports 0.20 ± 0.03. For Load, the seed-to-seed coefficient of variation is therefore about 29% for NMAE but only about 5% for MASE. The authors should state which aggregation rule they implemented, or correct the values. The mismatch sits in the row that carries the main claims, and it matters where margins fall inside the method's own seed spread, as with PV MASE at 0.65 ± 0.09 against 0.71 ± 0.03 and Net-load MASE at 0.50 ± 0.08 against 0.51 ± 0.00. A related point is that if the metrics use the stated global normalization scales, NMAE and MASE rank methods in the same order within each task. Counting the two as one leaves 12 task-metric comparisons, of which EnergyCA-LLM wins 11.

The experiments do not separate the two mechanisms the paper claims, pretrained semantic encoding and dynamic residual control. The paper also gives too little detail to check its claims about forecast-origin availability.

First, by Eqs. 4 to 6, the semantic state e_i reaches the predictor only through the window code r_i, that is, through the rank and strength gates. The fixed shared residual adapter in Table 2, Panel C removes these gates. As described, this also cuts the predictor off from the semantic state, so Panel C cannot show the effect of dynamic residual control while the semantic information is kept. Supporting the claim about state-dependent control needs a variant that keeps a fixed adapter and passes e_i through another route, such as an extra token, FiLM or concatenation.

Second, the headline 63.0% reduction is measured against No context, whose four-task mean NMAE is 0.414. This variant is weak on Load and Wind, and its Wind NMAE of 1.046 is worse than Persistence at 0.94. The four independent numerical models are also worse than Persistence on Wind at 1.043, and their Load NMAE of 0.132 is about 2.6 times that of TimeXer at 0.05. The gain over closer controls is smaller. Relative to the MLP schema encoder, for example, the four-task mean NMAE drops from 0.220 to 0.153. The variants are also not defined clearly enough to read the table. The shared numerical backbone in Panel A and No context in Panel B both appear to be single shared models without the semantic pathway, yet their Wind NMAE values are 0.412 and 1.046. The ablations include no shuffling control that gives the model context from another window of the same task, no smaller pretrained context encoder, and no analysis of the learned gates. As a result, it remains unclear how much of the gain over the MLP schema encoder comes from the pretrained semantics of Qwen3-8B.

Third, forecast-origin availability is the main motivation in Section 1, yet no experiment varies it. No fields are masked or delayed at test time, no ablation switches off only the availability block, and the controlled field removal and counterfactual-style sensitivity edits that lines 182 to 185 say the construction supports are not reported. The paper also does not list, for each benchmark, the context fields with their source-status tags, the time window and thresholds of each regime descriptor, or a literal serialized example. For instance, it does not say whether horizon temperature for GEFCom2014 is the observed value or a forecast. Nor does it say which weather information enters the Wind context, although Zhou et al. (2024) list weather information among the context factors recorded in SDWPF. Appendix A.2 states the right field rules, but readers need to see them applied to each benchmark. This matters most on Wind, where No context keeps the target history and exogenous numerical channels yet raises NMAE from 0.253 to 1.046.

Questions for authors
  1. Which aggregation rule produced NMAE and MASE for each task, and did EnergyCA-LLM and the baselines use the same rule and the same global normalization scales? Please give unrounded per-seed NMAE and MASE for the EnergyCA-LLM rows, and explain or correct the standard-deviation mismatch described above.
  2. In the fixed shared residual adapter variant, does e_i reach the model by any route? If not, can you report a variant with a fixed adapter that still passes e_i to the model through a route other than the rank and strength gates?
  3. Please state exactly what each Table 2 variant contains, that is, which of the task and cadence embeddings, the availability masks, the controller and the gated low-rank adapters it keeps. In particular, why is Wind NMAE 0.412 for the shared numerical backbone but 1.046 for No context?
  4. For each benchmark, can you give a table of context fields that lists each field's source-status tag, its availability at issuance time, the time window each regime descriptor summarizes and its descriptor thresholds, along with one literal serialized example? Please cover horizon temperature for Load and the weather inputs for Wind, and explain what Future-SCADA means in the SDWPF baseline.
  5. Under the common training budget of at most 2,560 optimizer updates, what batch size did each baseline use and how many updates did it actually run? Did the baselines and the four independent numerical models converge on validation? I ask because the four independent numerical models are worse than Persistence on Wind. Were FusionSF and the SDWPF Future-SCADA predictor retrained under this budget?
  6. Net-load forecasts are issued at 00:00 with lead times from 22 to 47.5 hours at half-hourly cadence. That gives 52 points, but H = 48. Which lead times are scored? Are the SDWPF targets at the turbine level or the farm level? For each task, please report the test dates, the forecast origins and the number of test windows. The GEFCom2014 load track ran as rolling monthly rounds, so if Load evaluation uses only the Task 15 month, please discuss how much temporal coverage this gives.
  7. In Figure 2, which lead time does each point of the rolling forecast show?
Reviewer_9gwPOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes EnergyCA-LLM, a unified forecasting framework for four energy forecasting tasks. The method employs a frozen pretrained language model to encode contextual information into semantic representations, which are then combined with numerical representations to dynamically control low-rank residual adapters within a shared forecasting model. The proposed method has practical value for engineering applications.

Critical strengths

EnergyCA-LLM unifies load, wind power, photovoltaic generation, and net-load forecasting within a single shared model and combines textual semantics, historical numerical states, and field availability to generate window-specific modulation parameters.

Distinguishing among static information, historical observations, weather forecasts, and known-future information helps clarify the information boundary at the forecast origin.

Critical weaknesses
  1. The paper has some practical value for engineering applications. However, the proposed framework primarily integrates existing techniques, including pretrained language models, semantic–numerical feature fusion, and low-rank residual adaptation, while its methodological novelty remains insufficiently clear. The authors are encouraged to further clarify the substantive differences between their approach and existing related methods and explicitly identify the core novel contributions of this work.

  2. Although the experiments cover four energy forecasting tasks, only one dataset is used for each task. This experimental setting is insufficient to demonstrate whether the proposed framework can consistently achieve performance improvements across different datasets within the same task category.

  3. The paper only employs Qwen3-8B as the pretrained language model and lacks comparisons examining how different LLMs affect forecasting performance. Furthermore, although the authors provide relevant ablation studies, the individual contributions of different gating components within the dynamic residual adaptation mechanism have not been thoroughly investigated. Consequently, it remains unclear how each component influences the final forecasting performance.

Questions for authors

Q.1 I find the research objective of this paper somewhat unclear. The authors develop a unified forecasting framework by jointly training on four different energy forecasting tasks, but what specific capability is this setting primarily intended to evaluate? The paper emphasizes cross-task knowledge sharing and task-context-aware dynamic adaptation, yet the current experiments do not sufficiently distinguish the contributions of these two mechanisms to forecasting performance, leaving the additional benefits of unified modeling unclear. Furthermore, if the paper primarily aims to improve forecasting accuracy, the current baseline comparisons remain insufficiently comprehensive. The authors are encouraged to consider additional representative recent methods for unified time-series forecasting.

My rating is primarily based on the concern raised in Weakness 1.

Reviewer_jLPYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies unified short-term forecasting for load, wind, PV, and net load, where a single checkpoint must serve all four tasks. The proposed EnergyCA-LLM encodes the forecast-origin context with a frozen Qwen3-8B, fuses this semantic state with numerical trajectory features under availability masks, and uses window-specific gates to modulate shared low-rank residual adapters. On four public benchmarks, one shared checkpoint per seed reports the lowest error in 15 of 16 task–metric comparisons against twelve baselines, losing only on PV RMSE. Three ablations, on unified training, encoder replacement, and adapter control, are used to justify parameter sharing, semantic encoding, and dynamic adaptation respectively.

Critical strengths
  1. The availability-aware schema is the main contribution. It records what information exists at the forecast origin and enforces that boundary in both input streams, removing a leakage ambiguity that applied energy forecasting work often leaves implicit. What makes this convincing is the encoder-replacement ablation, where the pretrained semantic state improves on both no context and a learned MLP encoder.

  2. The evaluation is set up carefully. One shared checkpoint per seed is tested across four quite different tasks under matched protocols, and each ablation panel maps to one design claim. This lets the gains be credited to the proposed components rather than to per-task tuning, which is the main reason I weigh the evidence positively.

Critical weaknesses
  1. The comparison set does not support the headline claim. The paper argues for unified forecasting with one shared checkpoint, yet none of the shared-checkpoint forecasters cited in Section 2.1, namely UniTS, Moirai, TimesFM, MOMENT, Time-MoE, and Timer-XL, appears in Table 1. And where the included baselines are strongest, the margins are close to ties, with PV RMSE slightly worse than FusionSF. The central advantage over the most related line of work is therefore unquantified.

  2. The mechanism that motivates the framework is never directly tested. Section 4.1 says the schema supports controlled field removal, shuffling, and sensitivity edits, but no such experiment appears in Section 5 or the appendix. The case for the core idea currently rests on dataset-level differences, which is considerably weaker evidence than one controlled manipulation of availability would provide.

Questions for authors
  1. Please explain why the forecasting models discussed in Section 2.1, including TimesFM, Moirai, UniTS, MOMENT, Time-MoE, and Timer-XL, are absent from Table 1. Clarify whether they were considered under the same evaluation protocol and how baseline hyperparameters were selected within the common training budget.

  2. PV sMAPE differs substantially despite similar NMAE, reaching 112.49 for PatchTST, 36.84 for FusionSF, and 31.70 for EnergyCA-LLM. Please specify the sMAPE formula, aggregation procedure, and treatment of zero denominators.

  3. Section 4.1 describes controlled field removal, shuffling, and counterfactual sensitivity edits. Please provide any results from these tests or clarify what evidence, beyond differences across datasets, supports the availability-aware mechanism.

Comments on the paper

1 comment

Dear Reviewers,

Please read authors' rebuttals and confirm if your concerns are addressed well (if you have not done so). You can double check if all concern in your reviews are addressed or not, which will be the key things we need to discuss later.

Best regards,

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.