- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes EnergyCA-LLM, one shared model that forecasts load, wind power, PV generation and net load, each at its native cadence and horizon. The targets are hourly load from GEFCom2014 Load Task 15, 10-minute wind power from SDWPF, hourly PV generation from MMSP/FusionSF, and half-hourly regional net load from the Great Britain GSP-group benchmark. For each window, a deterministic constructor turns the information available at the forecast origin into text. The schema covers task, asset, geography, issuance time, calendar, weather, recent operation and field availability. Continuous weather and operating variables stay in the numerical stream, and the text carries low-cardinality regime descriptors defined on the training split. A frozen Qwen3-8B context encoder reads this text, and masked pooling followed by a trainable projection gives a semantic state. The numerical backbone separates endogenous and exogenous inputs as in TimeXer and uses cross-attention to forecast-known covariates. The controller pools the unmodulated base state, embeds field availability, and combines them with the semantic state into a window code. This code sets the rank and strength gates of shared low-rank adapters in each selected layer. A shared decoder with task- and cadence-conditioned queries produces the forecasts. Training uses a task-balanced, scale-normalized Huber loss plus a residual penalty on the low-rank corrections. Under a common training budget of at most 2,560 optimizer updates and three seeds, the paper reports the lowest error in 15 of the 16 task-metric comparisons against twelve baselines, using NMAE, RMSE, sMAPE and MASE. The one exception is PV RMSE, where FusionSF reaches 0.099877 and EnergyCA-LLM 0.100042. Table 2 reports unified-training, encoder-replacement and adapter-control ablations, which compare independent and shared numerical models, several replacements for the context encoder, and a fixed shared residual adapter.
Critical strengths
The paper makes explicit what is known at the forecast origin. It defines an information set at issuance time, a field-level availability mask, four source-status tags named static, observed, forecast and known-future, and regime descriptors defined from training data, with descriptor thresholds fixed before validation and testing. These are set out in Section 3, Section 4.1 and Appendix A.2. One shared checkpoint per seed serves all four task families and keeps the native cadences, channel sets and horizons of each. Trajectory prediction stays in the numerical model, and the frozen LLM only encodes context to control adaptation. This is a reasonable answer to the caution the authors cite from Tan et al. (2024), that some gains in LLM-enhanced forecasters arise from architecture and training rather than language-specific capabilities. The setting matters for grid operation, and it is the main reason I see value in this work.
The comparison covers four public benchmarks and four metrics. It includes general numerical and LLM-enhanced baselines as well as task-specific references, namely Load-GAM, the SDWPF Future-SCADA predictor, Clear-Sky, FusionSF and a GAM for net load. Results use three seeds, and Table 2 has three ablation panels. Some margins are large. Load sMAPE falls from 5.16 to 3.11, Wind sMAPE from 96.81 to 35.09, and Wind NMAE from 0.31 to 0.25. The authors also report their loss on PV RMSE openly. If the evaluation protocol holds up, these results would be useful to practitioners.
Critical weaknesses
Appendix A.3 states that when each task uses one global normalization scale for each metric, fixed across methods and seeds, NMAE = A/U and MASE = A/S. Then MASE = c × NMAE with c = U/S, so the seed mean and the population standard deviation both scale by c. The Table 1 means fit a single c per task across all rows within rounding, with c at about 6.27 to 6.65 for Load, 0.770 to 0.782 for Wind, 2.455 to 2.483 for PV and 7.70 to 7.79 for Net load. The standard deviations shown for every baseline also fit this ratio within rounding. The EnergyCA-LLM row does not. Table 2 gives three-decimal NMAE values from the same checkpoints. On Load, 0.031 ± 0.009 implies a MASE standard deviation of about 0.053 to 0.063, but Table 1 reports 0.20 ± 0.01. On PV, 0.264 ± 0.009 implies about 0.021 to 0.024, but Table 1 reports 0.65 ± 0.09. On Net load, 0.063 ± 0.003 implies about 0.019 to 0.027, but Table 1 reports 0.50 ± 0.08. On Wind, 0.253 ± 0.026 implies about 0.020 to 0.021, but Table 1 reports 0.20 ± 0.03. For Load, the seed-to-seed coefficient of variation is therefore about 29% for NMAE but only about 5% for MASE. The authors should state which aggregation rule they implemented, or correct the values. The mismatch sits in the row that carries the main claims, and it matters where margins fall inside the method's own seed spread, as with PV MASE at 0.65 ± 0.09 against 0.71 ± 0.03 and Net-load MASE at 0.50 ± 0.08 against 0.51 ± 0.00. A related point is that if the metrics use the stated global normalization scales, NMAE and MASE rank methods in the same order within each task. Counting the two as one leaves 12 task-metric comparisons, of which EnergyCA-LLM wins 11.
The experiments do not separate the two mechanisms the paper claims, pretrained semantic encoding and dynamic residual control. The paper also gives too little detail to check its claims about forecast-origin availability.
First, by Eqs. 4 to 6, the semantic state e_i reaches the predictor only through the window code r_i, that is, through the rank and strength gates. The fixed shared residual adapter in Table 2, Panel C removes these gates. As described, this also cuts the predictor off from the semantic state, so Panel C cannot show the effect of dynamic residual control while the semantic information is kept. Supporting the claim about state-dependent control needs a variant that keeps a fixed adapter and passes e_i through another route, such as an extra token, FiLM or concatenation.
Second, the headline 63.0% reduction is measured against No context, whose four-task mean NMAE is 0.414. This variant is weak on Load and Wind, and its Wind NMAE of 1.046 is worse than Persistence at 0.94. The four independent numerical models are also worse than Persistence on Wind at 1.043, and their Load NMAE of 0.132 is about 2.6 times that of TimeXer at 0.05. The gain over closer controls is smaller. Relative to the MLP schema encoder, for example, the four-task mean NMAE drops from 0.220 to 0.153. The variants are also not defined clearly enough to read the table. The shared numerical backbone in Panel A and No context in Panel B both appear to be single shared models without the semantic pathway, yet their Wind NMAE values are 0.412 and 1.046. The ablations include no shuffling control that gives the model context from another window of the same task, no smaller pretrained context encoder, and no analysis of the learned gates. As a result, it remains unclear how much of the gain over the MLP schema encoder comes from the pretrained semantics of Qwen3-8B.
Third, forecast-origin availability is the main motivation in Section 1, yet no experiment varies it. No fields are masked or delayed at test time, no ablation switches off only the availability block, and the controlled field removal and counterfactual-style sensitivity edits that lines 182 to 185 say the construction supports are not reported. The paper also does not list, for each benchmark, the context fields with their source-status tags, the time window and thresholds of each regime descriptor, or a literal serialized example. For instance, it does not say whether horizon temperature for GEFCom2014 is the observed value or a forecast. Nor does it say which weather information enters the Wind context, although Zhou et al. (2024) list weather information among the context factors recorded in SDWPF. Appendix A.2 states the right field rules, but readers need to see them applied to each benchmark. This matters most on Wind, where No context keeps the target history and exogenous numerical channels yet raises NMAE from 0.253 to 1.046.
Questions for authors
- Which aggregation rule produced NMAE and MASE for each task, and did EnergyCA-LLM and the baselines use the same rule and the same global normalization scales? Please give unrounded per-seed NMAE and MASE for the EnergyCA-LLM rows, and explain or correct the standard-deviation mismatch described above.
- In the fixed shared residual adapter variant, does e_i reach the model by any route? If not, can you report a variant with a fixed adapter that still passes e_i to the model through a route other than the rank and strength gates?
- Please state exactly what each Table 2 variant contains, that is, which of the task and cadence embeddings, the availability masks, the controller and the gated low-rank adapters it keeps. In particular, why is Wind NMAE 0.412 for the shared numerical backbone but 1.046 for No context?
- For each benchmark, can you give a table of context fields that lists each field's source-status tag, its availability at issuance time, the time window each regime descriptor summarizes and its descriptor thresholds, along with one literal serialized example? Please cover horizon temperature for Load and the weather inputs for Wind, and explain what Future-SCADA means in the SDWPF baseline.
- Under the common training budget of at most 2,560 optimizer updates, what batch size did each baseline use and how many updates did it actually run? Did the baselines and the four independent numerical models converge on validation? I ask because the four independent numerical models are worse than Persistence on Wind. Were FusionSF and the SDWPF Future-SCADA predictor retrained under this budget?
- Net-load forecasts are issued at 00:00 with lead times from 22 to 47.5 hours at half-hourly cadence. That gives 52 points, but H = 48. Which lead times are scored? Are the SDWPF targets at the turbine level or the farm level? For each task, please report the test dates, the forecast origins and the number of test windows. The GEFCom2014 load track ran as rolling monthly rounds, so if Load evaluation uses only the Task 15 month, please discuss how much temporal coverage this gives.
- In Figure 2, which lead time does each point of the rolling forecast show?
Dear Reviewers,
Please read authors' rebuttals and confirm if your concerns are addressed well (if you have not done so). You can double check if all concern in your reviews are addressed or not, which will be the key things we need to discuss later.
Best regards,
Your AC