acceptodds
Under review as a conference paper at ICLR 2027

EU-CRD: Responsibility-Guided Credit Assignment for Forecast-Aware Carbon Scheduling

Abstract

Geo-distributed datacentres can reduce carbon emissions by shifting delay-tolerant jobs across time and space to better match renewable supply. However, such spatial-temporal scheduling depends on weather-dependent energy forecasts, where inevitable forecasting errors yield suboptimal decisions or even mislead the scheduler. Conventional reinforcement learning learns from returns that reflect both routing quality and forecast quality, without an explicit mechanism to isolate their contributions. EU-CRD (Epistemic Uncertainty Counterfactual Responsibility Decomposition) addresses this credit-assignment problem by constructing a hindsight forecast regret proxy and a policy-action value gap, modulating the latter with critic ensemble disagreement, and using their relative magnitudes to reweight the routing advantage during training. No additional responsibility computation is required at deployment. In a five-datacentre simulation driven by measured wind, paired continuation runs from three shared checkpoints compare EU-CRD with a matched control retaining the same backbone, critic ensemble, responsibility computation and auxiliary training. Under deterministic decoding on six development windows, EU-CRD reduces carbon emissions by with a degraded forecast and by with a perfect forecast. The same models also reduce carbon by and , respectively, on six confirmation windows from a separate year. All three paired seeds favour EU-CRD under both forecast settings in both evaluations, with all service requirements met. On the development windows, alternative objectives and component ablations also yield higher carbon emissions under degraded forecasts for every seed. These results support responsibility-guided reweighting as a training-only mechanism for forecast-aware carbon scheduling.

Primary area:
reinforcement learning
Keywords:
carbon-aware scheduling, forecast-aware reinforcement learning, credit assignment, counterfactual reasoning, uncertainty-aware reinforcement learning, datacentre task scheduling, renewable energy, PPO

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_qeQfOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper examines data center electricity consumption optimization. The idea is that data centers may postpone work or relocate work to times or places where more green energy is available, thereby reducing their overall carbon footprint. As the authors note, to successfully solve this problem, any system has to rely on forecasts: both in terms of localized weather (and thereby the green energy available) and the incoming jobs. Since these forecasts may be imprecise, this hinders learning; a standard RL agent (aka PPO) will, for example, not separate whether a produced action was due to its own bad policy or due to a poorly given observation. To this end, the authors propose a method of separating these two errors and adjusting for the observation error within the advantage estimation of PPO. The authors then demonstrate how their method improves over an agent not adjusting for the forecast error.

Critical strengths

This idea is quite neat and more widely applicable than just data centers, as many systems require forecasts where RL agents need to be able to deal with their uncertainties. Commonly, I would think this work falls under robust RL, where we train under pessimistic assumptions. I am not too familiar with that field, and the authors also mention it in their related work. Generally though, I had not seen such an explicit separation and correction before on these kinds of forecasts.

Critical weaknesses

Scale and validity of the experiments

If I infer it correctly, as per the first paragraph of 4.1, an environment episode is a two-hour time span in which the 5 data centers have to process 35 jobs. As far as I can tell, this setting always repeats, resulting in quite a small problem. Total training timesteps is reported to be 112000, which is also on the low side for RL problems. Additionally, results are only reported over 3 seeds. In Figure 1, one of the dots of perfect information sits above the site-mean forecast, which I believe shouldn't happen if the policy has been trained well. Overall, this seems like a small amount of experiments & repetitions with a policy that hasn't trained well yet.

On top of that, when we take a look at the absolute numbers (e.g., Table 1), we see improvements in the order of 0.01-0.02 kg CO2, "pooled" (summed) over 3 seeds, so roughly 0.003-0.01 kg CO2 per seed. As I learned, this is roughly equivalent to a phone charge or less. Essentially then, we are measuring data center electricity consumption on the level of detail of a human breathing for a couple of minutes. I can't help but think that we are then mostly measuring noise.

General presentation Various things are unclear, not introduced or introduced after they are already used:

  • Pooled is used as the term here with which data of various seeds is aggregated, but reading the appendix it becomes clear that the data is summed over the seeds. This is quite nonstandard and also inflates CO2 numbers here.
  • The term "arm" is never fully introduced but has to be inferred
  • "2021 development windows" and "2020 confirmation windows" are not introduced but assumed to be known objects. Are these datasets?
  • If the above are not datasets, where does the data come from or is it synthetic (e.g. for the jobs and the available green power)
  • It is not made clear what an episode exactly is and how much they vary.
  • The policy is described as a joint policy (line 155), but the local models also have their own observations and operate independently. As a result, is it not a joint policy? The multi-agent setup here is unclear.
  • Line 083-0-88: hard to read this sentence
  • Line 363: " All reported cells satisfy the service requirements.": no reference here to anything; are 'the reported cells' the cells in all tables?

Further notes:

  • No clear baseline from other literature given
Questions for authors

I welcome the authors to challenge my criticisms above and let me know if I wildly misunderstood bits of the experiments. I have no further questions.

Reviewer_y6ZXOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes EU-CRD, a responsibility-guided credit-assignment method for carbon-aware datacenter scheduling under imperfect renewable-energy forecasts. The key idea is to separate forecast-related and routing-related responsibility using a hindsight forecast-regret proxy, a counterfactual action-value gap, and critic-ensemble uncertainty, and then use the resulting responsibility score to reweight the PPO advantage during training. The responsibility computation is used only during training and does not add deployment overhead.

Experiments on a five-datacenter simulator report lower carbon emissions than a matched control under both attenuated and perfect forecasts, with additional cross-year evaluation on wind traces from a separate year.

Critical strengths

The paper addresses an important practical problem: forecast errors can substantially affect carbon-aware scheduling decisions.

The matched-control design is useful: the control retains the same backbone, critic ensemble, auxiliary training, and compute budget, helping isolate the effect of the proposed advantage reweighting.

The paper includes component ablations, alternative training objectives, and a cross-year confirmation experiment. The training-only design also avoids additional deployment-time responsibility computation.

Critical weaknesses
  1. The necessity of explicit responsibility decomposition is not sufficiently justified. A simpler formulation is to treat the forecast as an input and train the routing policy directly on the downstream carbon/SLA objective: fixed forecast --> routing policy -> task reward. Standard policy learning should in principle learn how much to rely on forecast information from task-level returns. A particularly important missing baseline is standard PPO trained over the same, or preferably randomized, forecast-error distribution. The current matched control is exposed only to one fixed attenuation process, and the authors explicitly note that domain randomization is not evaluated.

  2. A task-based end-to-end learning alternative is also missing. If the forecasting module is trainable, prediction and decision can be optimized directly for the downstream task objective rather than introducing manually designed responsibility proxies. This general principle has been studied in task-based end-to-end learning, e.g., Donti et al. (2017) "Task-based End-to-end Model Learning in Stochastic Optimization", where prediction is optimized according to downstream decision quality rather than prediction accuracy alone. The paper should better motivate when EU-CRD is preferable to such predict-then-optimize/end-to-end formulations.

  3. The proposed “responsibility” decomposition is heuristic and is not validated as actual responsibility. The authors explicitly state that the forecast-regret quantity is not the realized action's causal loss or a proven bound, and that the resulting shares are weighting heuristics rather than an additive responsibility decomposition. Moreover, EU-CRD improves carbon by 10.1% even under a perfect forecast, where forecast error is absent. This raises the question of whether the method is mainly providing generic advantage reweighting/regularization rather than correctly separating forecast and action responsibility. The paper itself notes that the experiments do not isolate how much of the improvement is specific to forecast degradation.

  4. The forecast-error model and evaluation scope are limited. The primary degraded forecast is generated by amplitude attenuation while preserving the temporal shape and timing of the true renewable trajectory. This does not capture many realistic forecast errors such as timing shifts, missed ramps, bias, or errors from an actual learned forecaster. In addition, the evaluation uses five datacenters and a fixed workload of only 35 jobs. The cross-year experiment changes wind traces but keeps the same sites, workload, and forecast intervention; the authors acknowledge that it does not establish generalization to new workloads or forecast-error processes.

  5. The empirical robustness and baseline coverage are limited. Results use only three training seeds, and improvements are not uniform across seed-window combinations. More importantly, under sampled decoding the direction of improvement changes across decoding seeds, so the claimed benefit is restricted to deterministic decoding. In addition, the CVaR-, mean-variance-, and other risk-inspired comparisons are surrogate variants implemented within the authors' backbone rather than implementations of the corresponding published methods, which limits the strength of the baseline comparison.

Questions for authors
  1. Why is explicit forecast/action responsibility decomposition necessary rather than simply treating the forecast as an input and training the routing policy directly on the downstream task reward? Could you compare against standard PPO trained under randomized forecast errors?

  2. If the forecast module is trainable, how does EU-CRD compare with task-based end-to-end prediction/decision learning such as Donti et al. (2017)?

  3. Since the proposed responsibility scores are heuristic, what evidence shows that they actually measure forecast-vs-action responsibility rather than acting as a generic advantage-reweighting mechanism? In particular, how do you explain the substantial gain under the perfect-forecast setting?

  4. Could you evaluate on more realistic forecast errors, ideally predictions from an actual forecasting model, as well as different workload distributions and larger datacenter/job settings?

  5. Can you provide stronger statistical evaluation and comparisons with full implementations of relevant robust/risk-sensitive RL baselines?

Reviewer_VKWWOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper seems to address the challenge of disentangling endogenous vs. exogenous contributions to reward signals in RL policy gradients, especially in the context of carbon-aware datacenter job scheduling. Endogenous contributions to reward come from actions taken by the agent whereas exogenous contributions come from the environment itself that are not intrinsically tied to the the agent's actions. In the context of this paper, this paper tries to disentangle how good individual actions are by RL policy from the quality of the carbon intensity forecast that's provided as input to the state of the RL agent.

Critical strengths

The paper instantiates the well-known credit-assignment problem in RL in the context of carbon-aware job scheduling.

Critical weaknesses

My main assessment is that this paper is very hard to read. It reads as very AI-generated, which alone is not disqualifying. However, it is almost impossible to understand what this paper is doing. Furthermore, I cannot trust that the experiments actually implement what the paper is describing.

Just as some examples, it says, “carbon-aware benchmarks write the forecast straight into their reference agents’ observations.” What does that mean? For Figure 1, it is not clear at all where the data used to make the figure comes from. Without knowing where this data comes from, it is impossible for me to evaluate whether the numbers are meaningful. What does it mean for uncertainty-aware methods to “make caution conditional”? Around line 76, the paper asks, “Does such a weight lower deployment carbon”? What does “such a weight” mean?

Further, this lack of clarity does not just make the paper difficult to read. It also raises questions about the validity of any experiments. The reward function is not defined in the main text. Around line 167, the paper says that "The reward scalarises this objective as a per-step carbon penalty and a credit for completed compute" referring to Appendix K. But Appendix K says, “Five further terms exist in the implementation and are zero here.” Are there parts of the implementation that are not explained here? In the actual reward function given in Equation 12 in the appendix, there are also many magic numbers (constants that are not explained). This raises questions about how much I can trust the paper’s description to reflect what was actually implemented.

Questions for authors

Here is just a small sampling of undefined, poorly defined, or unexplained terms and phrases. There are too many to list them all:

  • line 81: what does it mean for a local agent to "pace" a datacenter's queue?
  • line 112: what does "online line" refer to?
  • line 149: what is DC?
  • line 150: κigreen\kappa_i^\text{green} and κibrown\kappa_i^\text{brown}
  • I cannot understand Figure 2 at all. The top graph mentions ef(t)e_f(t) which is not defined anywhere. I also don't know how to interpret the bottom part.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.