- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper evaluates learned policies for power-system balancing along three dimensions: agreement with historical decisions, closed-loop behavior, and operational outcomes. Using Belgian grid data and a learned simulator, the authors find that changes in decision agreement and intervention frequency do not consistently translate into improved system balance, motivating evaluation beyond historical agreement alone.
Critical strengths
This work studies a practical problem: agreement with historical operators may not reflect a policy’s closed-loop performance in the power system.
Critical weaknesses
-
Insufficient methodological novelty. The proposed framework primarily distinguishes three evaluation dimensions, while the technical components largely use established methods. The paper does not clearly establish what new methodology or actionable guidance follows beyond reporting these dimensions separately.
-
Limited experimental scope. All experiments use a single TSO (Transmission System Operator) task, a three-action abstraction, and the same learned simulator. One-step predictive accuracy does not establish reliable counterfactual or long-horizon evaluation, leaving unclear whether the weak outcome differences reflect the system or the evaluator.
-
Incomplete coverage of related work. The paper omits directly relevant research on offline versus closed-loop evaluation (Codevilla et al., 2018), imitation-induced distribution shift (Ross et al., 2011), local label smoothing (Bahri & Jiang, 2021), and selective classification (Geifman & El-Yaniv, 2017). These omissions weaken its novelty positioning.
Reference
Codevilla et al. (2018) On Offline Evaluation of Vision-based Driving Models
Ross et al. (2011) A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Bahri & Jiang (2021) Locally Adaptive Label Smoothing for Predictive Churn
Geifman & El-Yaniv (2017) Selective Classification for Deep Neural Networks
Questions for authors
Please see the weakness part.
At the current stage, I don't think the paper meets the standard for acceptance at a top-tier ML conference, and I tend toward rejection. However, I am open to revisiting my assessment based on the authors’ response and subsequent discussion.
2 comments
Reply to Reviewer 7ycB (Comment 2 of 2)
W2: single TSO, three-action abstraction, one simulator; system or evaluator?
The weak outcome differences in the initial submission came from the evaluator, not from the system. The revision evaluates all policies with two evaluators on the same test days. The learned transition model fails the control policies. Under the grey-box evaluator, which passes them, clear differences in reserve use appear (Figure 2, Table 9). These differences rest on the evaluator's stated assumptions (Section 4). For the three-action abstraction, the activation volume is an explicit input of the evaluator, and every comparison keeps its direction across the volumes we tested (Table 17).
On the single TSO, we repeated the procedure on a second control area, the German control block, using only public data and acceptance criteria fixed before any result (Appendix A.13). The aim was to see whether the procedure carries over, and it does: the evaluator could be built and validated, and it reproduces German operation as well as Belgian.
The checks were also informative. Three of four control policies behaved as expected, but always activating upward did not increase automatic reserve activation, so the response check is not met there. A likely reason, found only afterwards, is that the German system was persistently short in the test months.
This is the validation step doing its job: it shows where the evaluator can be trusted and where not. For the policy comparison itself, Belgium is the more informative setting. Elia activates mFRR in about 15% of quarter-hours, against 0.36% in the German block, so the German data contain too few manual decisions to learn a policy from. We therefore present Germany as evidence that the procedure is portable, and report the policy results for Belgium. Testing the policy comparison on data from further control areas with frequent mFRR is part of our future work.
W3: related work
We added all four references to the related work (Section 1). Briefly, our work differs as follows.
- Codevilla et al. (2018) compare offline metrics with closed-loop driving performance in a simulator they can trust. For TSO balancing, no simulator of how the market and the reserves respond to a different decision policy is available as ground truth, so the evaluator itself must be validated first. Our protocol adds this step.
- Ross et al. (2011) show that imitation errors compound once a policy reaches states the expert never visited. We measure this drift as closed-loop behavior, rather than correcting it with expert queries, which are not available in TSO operation.
- Bahri and Jiang (2021) and Geifman and El-Yaniv (2017) are closely related to our ambiguity-aware supervision and selective deferral. They evaluate predictions one at a time. We ask what the changed or deferred decisions do when the policy acts in sequence.
Presentation (rated difficult to follow)
We revised the presentation throughout, while keeping the structure, research question and three contributions. If particular sections remain difficult to follow in the revised version, we would be glad to know which, and will revise them. We hope the revision addresses the reviewer's concerns, and we would be glad to discuss any remaining questions.
Reply to Reviewer 7ycB (Comment 1 of 2)
We thank the reviewer for the review and for being open to revisiting the assessment. The comments helped us make the contribution concrete. The revision offers an evaluation protocol with a validation step that can be tested (Section 2.3, Appendix A.9), shows with two evaluators why this step is needed (Section 2.3, Appendix A.9), checks the procedure on a second control area (Appendix A.13), and discusses the four suggested references (Section 1).
W1: methodological novelty and actionable guidance
The contribution is a validation protocol for offline decision support, and evidence that each of its steps is needed. Codevilla et al. (2018) showed that offline metrics can disagree with closed-loop performance in a simulator that can be trusted. In TSO balancing no such trusted simulator is available, so the protocol adds the step their setting did not need, testing the evaluator itself before comparing outcomes. On real TSO data, we show three gaps that standard validation overlooks.
First, agreement does not determine behavior. The learned policies agree with the operator on most decisions, yet intervene three to four times as often (Figure 2). Agreement stays high because both mostly hold, so it hides this difference.
Second, predictive accuracy does not mean that an evaluator responds to actions. Our learned transition model reached a one-step R² of 0.97, but could not tell the operator from always holding or random actions (Appendix A.9, Figure 7). Positive controls detect this failure and need only the expected direction of an effect.
Third, a responsive evaluator still gives no single verdict. The policies use less automatic reserve but more reserve in total (Section 3.3, Table 9).
The actionable guidance follows from these gaps. Evaluate agreement, behavior and outcome separately. Before comparing outcomes, show with simple control policies that the evaluator responds to actions. Report outcomes on several metrics that the action can physically change. The evaluator itself can be built from public data, as the German control block shows (Appendix A.13).