- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper examines data center electricity consumption optimization. The idea is that data centers may postpone work or relocate work to times or places where more green energy is available, thereby reducing their overall carbon footprint. As the authors note, to successfully solve this problem, any system has to rely on forecasts: both in terms of localized weather (and thereby the green energy available) and the incoming jobs. Since these forecasts may be imprecise, this hinders learning; a standard RL agent (aka PPO) will, for example, not separate whether a produced action was due to its own bad policy or due to a poorly given observation. To this end, the authors propose a method of separating these two errors and adjusting for the observation error within the advantage estimation of PPO. The authors then demonstrate how their method improves over an agent not adjusting for the forecast error.
Critical strengths
This idea is quite neat and more widely applicable than just data centers, as many systems require forecasts where RL agents need to be able to deal with their uncertainties. Commonly, I would think this work falls under robust RL, where we train under pessimistic assumptions. I am not too familiar with that field, and the authors also mention it in their related work. Generally though, I had not seen such an explicit separation and correction before on these kinds of forecasts.
Critical weaknesses
Scale and validity of the experiments
If I infer it correctly, as per the first paragraph of 4.1, an environment episode is a two-hour time span in which the 5 data centers have to process 35 jobs. As far as I can tell, this setting always repeats, resulting in quite a small problem. Total training timesteps is reported to be 112000, which is also on the low side for RL problems. Additionally, results are only reported over 3 seeds. In Figure 1, one of the dots of perfect information sits above the site-mean forecast, which I believe shouldn't happen if the policy has been trained well. Overall, this seems like a small amount of experiments & repetitions with a policy that hasn't trained well yet.
On top of that, when we take a look at the absolute numbers (e.g., Table 1), we see improvements in the order of 0.01-0.02 kg CO2, "pooled" (summed) over 3 seeds, so roughly 0.003-0.01 kg CO2 per seed. As I learned, this is roughly equivalent to a phone charge or less. Essentially then, we are measuring data center electricity consumption on the level of detail of a human breathing for a couple of minutes. I can't help but think that we are then mostly measuring noise.
General presentation Various things are unclear, not introduced or introduced after they are already used:
- Pooled is used as the term here with which data of various seeds is aggregated, but reading the appendix it becomes clear that the data is summed over the seeds. This is quite nonstandard and also inflates CO2 numbers here.
- The term "arm" is never fully introduced but has to be inferred
- "2021 development windows" and "2020 confirmation windows" are not introduced but assumed to be known objects. Are these datasets?
- If the above are not datasets, where does the data come from or is it synthetic (e.g. for the jobs and the available green power)
- It is not made clear what an episode exactly is and how much they vary.
- The policy is described as a joint policy (line 155), but the local models also have their own observations and operate independently. As a result, is it not a joint policy? The multi-agent setup here is unclear.
- Line 083-0-88: hard to read this sentence
- Line 363: " All reported cells satisfy the service requirements.": no reference here to anything; are 'the reported cells' the cells in all tables?
Further notes:
- No clear baseline from other literature given
Questions for authors
I welcome the authors to challenge my criticisms above and let me know if I wildly misunderstood bits of the experiments. I have no further questions.