acceptodds
Under review as a conference paper at ICLR 2027

Beyond Decision Agreement: Evaluating Reinforcement Learning Policy Behavior and Outcomes in Power-System Balancing

Abstract

Learned decision-support policies are often evaluated by their agreement with historical operator decisions. However, historical agreement does not reveal how a policy behaves when its decisions are executed sequentially, because each decision changes the state for subsequent decisions. We study this issue in offline reinforcement learning for power-system balancing at a Belgian transmission system operator (TSO). We evaluate learned policies using three complementary measures: , historical agreement with operator decisions; , behavior during closed-loop execution; and , the resulting system-level performance, measured by mean absolute Area Control Error (ACE). Across controlled changes to the reward, supervision, and decision-time deferral, we find that these measures can change independently. In particular, policies with similar historical agreement can produce substantially different closed-loop behavior, while deliberate improvements in agreement also alter closed-loop behavior. Yet, across all experiments, these behavioral differences correspond to only small, statistically undetectable changes in the resulting system imbalance. We conclude that historical decision agreement alone is insufficient to validate a learned decision-support policy. Instead, evaluating policy behavior alongside historical agreement and operational outcomes is essential for offline sequential decision support.

Primary area:
reinforcement learning
Keywords:
Offline Reinforcement Learning, Decision Support Systems, Uncertainty-Aware Decision Making, Closed-Loop Policy Evaluation, Transmission System Operators, Power-System Balancing

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_7ycBOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper evaluates learned policies for power-system balancing along three dimensions: agreement with historical decisions, closed-loop behavior, and operational outcomes. Using Belgian grid data and a learned simulator, the authors find that changes in decision agreement and intervention frequency do not consistently translate into improved system balance, motivating evaluation beyond historical agreement alone.

Critical strengths

This work studies a practical problem: agreement with historical operators may not reflect a policy’s closed-loop performance in the power system.

Critical weaknesses
  1. Insufficient methodological novelty. The proposed framework primarily distinguishes three evaluation dimensions, while the technical components largely use established methods. The paper does not clearly establish what new methodology or actionable guidance follows beyond reporting these dimensions separately.

  2. Limited experimental scope. All experiments use a single TSO (Transmission System Operator) task, a three-action abstraction, and the same learned simulator. One-step predictive accuracy does not establish reliable counterfactual or long-horizon evaluation, leaving unclear whether the weak outcome differences reflect the system or the evaluator.

  3. Incomplete coverage of related work. The paper omits directly relevant research on offline versus closed-loop evaluation (Codevilla et al., 2018), imitation-induced distribution shift (Ross et al., 2011), local label smoothing (Bahri & Jiang, 2021), and selective classification (Geifman & El-Yaniv, 2017). These omissions weaken its novelty positioning.

Reference

Codevilla et al. (2018) On Offline Evaluation of Vision-based Driving Models

Ross et al. (2011) A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning

Bahri & Jiang (2021) Locally Adaptive Label Smoothing for Predictive Churn

Geifman & El-Yaniv (2017) Selective Classification for Deep Neural Networks

Questions for authors

Please see the weakness part.

At the current stage, I don't think the paper meets the standard for acceptance at a top-tier ML conference, and I tend toward rejection. However, I am open to revisiting my assessment based on the authors’ response and subsequent discussion.

2 comments
AuthorsOpenReview ↗

Reply to Reviewer 7ycB (Comment 1 of 2)

We thank the reviewer for the review and for being open to revisiting the assessment. The comments helped us make the contribution concrete. The revision offers an evaluation protocol with a validation step that can be tested (Section 2.3, Appendix A.9), shows with two evaluators why this step is needed (Section 2.3, Appendix A.9), checks the procedure on a second control area (Appendix A.13), and discusses the four suggested references (Section 1).

W1: methodological novelty and actionable guidance

The contribution is a validation protocol for offline decision support, and evidence that each of its steps is needed. Codevilla et al. (2018) showed that offline metrics can disagree with closed-loop performance in a simulator that can be trusted. In TSO balancing no such trusted simulator is available, so the protocol adds the step their setting did not need, testing the evaluator itself before comparing outcomes. On real TSO data, we show three gaps that standard validation overlooks.

First, agreement does not determine behavior. The learned policies agree with the operator on most decisions, yet intervene three to four times as often (Figure 2). Agreement stays high because both mostly hold, so it hides this difference.

Second, predictive accuracy does not mean that an evaluator responds to actions. Our learned transition model reached a one-step R² of 0.97, but could not tell the operator from always holding or random actions (Appendix A.9, Figure 7). Positive controls detect this failure and need only the expected direction of an effect.

Third, a responsive evaluator still gives no single verdict. The policies use less automatic reserve but more reserve in total (Section 3.3, Table 9).

The actionable guidance follows from these gaps. Evaluate agreement, behavior and outcome separately. Before comparing outcomes, show with simple control policies that the evaluator responds to actions. Report outcomes on several metrics that the action can physically change. The evaluator itself can be built from public data, as the German control block shows (Appendix A.13).

AuthorsOpenReview ↗

Reply to Reviewer 7ycB (Comment 2 of 2)

W2: single TSO, three-action abstraction, one simulator; system or evaluator?

The weak outcome differences in the initial submission came from the evaluator, not from the system. The revision evaluates all policies with two evaluators on the same test days. The learned transition model fails the control policies. Under the grey-box evaluator, which passes them, clear differences in reserve use appear (Figure 2, Table 9). These differences rest on the evaluator's stated assumptions (Section 4). For the three-action abstraction, the activation volume is an explicit input of the evaluator, and every comparison keeps its direction across the volumes we tested (Table 17).

On the single TSO, we repeated the procedure on a second control area, the German control block, using only public data and acceptance criteria fixed before any result (Appendix A.13). The aim was to see whether the procedure carries over, and it does: the evaluator could be built and validated, and it reproduces German operation as well as Belgian.

The checks were also informative. Three of four control policies behaved as expected, but always activating upward did not increase automatic reserve activation, so the response check is not met there. A likely reason, found only afterwards, is that the German system was persistently short in the test months.

This is the validation step doing its job: it shows where the evaluator can be trusted and where not. For the policy comparison itself, Belgium is the more informative setting. Elia activates mFRR in about 15% of quarter-hours, against 0.36% in the German block, so the German data contain too few manual decisions to learn a policy from. We therefore present Germany as evidence that the procedure is portable, and report the policy results for Belgium. Testing the policy comparison on data from further control areas with frequent mFRR is part of our future work.

W3: related work

We added all four references to the related work (Section 1). Briefly, our work differs as follows.

  • Codevilla et al. (2018) compare offline metrics with closed-loop driving performance in a simulator they can trust. For TSO balancing, no simulator of how the market and the reserves respond to a different decision policy is available as ground truth, so the evaluator itself must be validated first. Our protocol adds this step.
  • Ross et al. (2011) show that imitation errors compound once a policy reaches states the expert never visited. We measure this drift as closed-loop behavior, rather than correcting it with expert queries, which are not available in TSO operation.
  • Bahri and Jiang (2021) and Geifman and El-Yaniv (2017) are closely related to our ambiguity-aware supervision and selective deferral. They evaluate predictions one at a time. We ask what the changed or deferred decisions do when the policy acts in sequence.

Presentation (rated difficult to follow)

We revised the presentation throughout, while keeping the structure, research question and three contributions. If particular sections remain difficult to follow in the revised version, we would be glad to know which, and will revise them. We hope the revision addresses the reviewer's concerns, and we would be glad to discuss any remaining questions.

Reviewer_P6XYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the evaluation of learned decision-support policies for power-system balancing using operational data from a Belgian TSO. The paper argues that historical agreement with operator decisions is not sufficient to evaluate a policy, since different decisions can lead to different subsequent system states and closed-loop behavior. The authors therefore consider three evaluation dimensions: Decision Agreement, Offline Closed-Loop Behavior, and Operational Outcome. Several offline learning methods are evaluated under controlled reward changes, ambiguity-aware supervision, and selective deferral. The results show that Decision Agreement and closed-loop intervention behavior can change substantially, while the measured Operational Outcome, mainly mean absolute ACE, changes only slightly.

Critical strengths
  1. The paper clearly motivates an important evaluation issue for sequential decision-support problems for safety-critical power systems.
  2. The paper uses real operational data from a TSO and provides several experiments, including reward modification, ambiguity-aware supervision, and selective deferral, to study the relationship among Decision Agreement, Closed-Loop Behavior, and Operational Outcome.
Critical weaknesses

The validity of the learned transition model for closed-loop evaluation is not sufficiently demonstrated. The transition model is trained from historical operator actions, whereas the closed-loop evaluation requires predicting the consequences of actions that may differ from those taken historically. It is therefore unclear whether the model can accurately capture long-horizon and counterfactual system responses. In addition, during Stage 2, every historical transition is treated as terminal, so the TD target reduces to the immediate reward and the learning problem becomes closer to an offline contextual-bandit problem than sequential offline RL. As a result, sequential effects are introduced only during evaluation through the learned transition model, rather than being considered during policy learning. This weakens the connection between the proposed learning methodology and the central motivation of the paper.

This concern is further amplified by the simplified action representation: the policy only chooses HOLD, UP, or DOWN, while the actual mFRR activation volume varies substantially across events. Since the impact on ACE should depend not only on the activation direction but also on the amount of reserve activated, it is unclear whether this abstraction is sufficient for evaluating operational outcomes.

Questions for authors
  1. How is the learned transition model validated for long-horizon closed-loop simulation? Can the authors provide multi-step prediction results in addition to the reported one-step (R^2)?

  2. Since the transition model is trained from historical operator actions, how can the model accuracy be verified for counterfactual actions that differ from the historical decision?

  3. The policy only determines the direction of mFRR activation, while the actual activation volume varies significantly. How is this variation reflected in the learned transition dynamics? Would explicitly including activation volume as part of the action provide a more accurate evaluation of ACE?

  4. Can the authors provide additional Operational Outcome metrics beyond mean (|ACE|), such as high-percentile or maximum ACE, threshold violations, and reserve activation cost? These may better capture differences among the policies.

3 comments
AuthorsOpenReview ↗

Reply to Reviewer P6XY (Comment 1 of 3)

We thank the reviewer for the detailed review and for emphasising the importance of the evaluation question and the value of real TSO data. The main concern, whether our evaluator can be trusted for actions the operator did not take, was investigated in depth in the revision. It is addressed with a grey-box evaluator (Section 2.3). We validate it against historical operation and with deliberately poor control policies (Appendix A.9), compare it with minute-level data (Appendix A.10), and test its main assumption by asking how strongly the market would have to react to our policies to change the conclusions (Appendix A.11). The reserve-use results are in Section 3.3, Figure 2 and Table 9.

W1 (a): validity of the transition model for long-horizon and counterfactual responses

The revision shows this explicitly instead of assuming it. For outcome evaluation, a grey-box evaluator built from the TSO's settlement accounting replaces the learned transition model of the initial submission (Section 2.3). We check it in three steps and state what each step establishes.

Long horizon. With the operator's actual actions, the evaluator reproduces full 96-quarter days of automatic reserve activation, against criteria we fixed before running it (Table 14, Figure 6). It is validated for the period after Belgium joined MARI, the European mFRR platform, which contains the whole test set.

Response to actions. The evaluator ranks every deliberately poor policy as worse than the operator, so it responds to actions in the right direction (Appendix A.9, Figure 7). The size of that response is close to Elia's minute-level data, which are analysed without our fitted equation (Appendix A.10, Table 16).

Counterfactual response. For actions the operator never took, no evaluator built from historical data, learned or physical, can be checked against observed outcomes. What can be done is to name the remaining assumptions and test how much they matter. The main one is that the system imbalance does not react to the policy. The conclusions hold unless market reactions cancel almost all of each extra activation (κ ≈ −0.95, Appendix A.11, Figure 9). The data point the other way. In the training data, the imbalance moves with the previous quarter's mFRR with a coefficient of +0.30. The activation volume and the efficacy of mFRR are varied as well, with the same conclusions (Appendix A.11, Table 17).

The protocol therefore gives an evaluator whose fidelity and response to actions are tested, and whose remaining assumptions are stated and stress-tested. This is a stronger basis than predictive accuracy alone. The learned model of the initial submission had excellent one-step accuracy and still failed the response test. This is the practical point of the paper: without such checks, a convincing evaluator can give uninformative results.

W1 (b): terminal transitions

Section 2.2 states this explicitly. With terminal transitions, Stage 2 is offline contextual-bandit learning, and Q(s,a) estimates the expected immediate reward (Eqs. 6–7). This is a deliberate choice, and the work remains reinforcement learning for three reasons.

First, the policies are learned from rewards, not from labels, with offline RL algorithms (CQL, AWAC, CRR). Their conservative or advantage-weighted objectives matter even with one-step targets, because they keep the policy close to actions supported by the data. The contextual bandit is the one-step case of offline RL, not a different problem, and changing the reward changes the learned behavior consistently across methods and seeds (Figure 1).

Second, sequential effects enter at evaluation by design. The paper asks what happens when policies trained from historical data act in sequence, because each decision changes the state that the next decision sees. This is exactly what agreement with historical decisions cannot capture, and it holds however the training targets were computed.

Third, one-step targets fit the physics of this task and do not drive our results. Most of the effect of an mFRR activation on the automatic reserves appears within the same quarter (Section 2.2, Appendix A.9). When we retrained CQL and AWAC with bootstrapped multi-step targets, decisions and outcomes barely changed (Appendix A.14).

AuthorsOpenReview ↗

Reply to Reviewer P6XY (Comment 2 of 3)

W2 / Q3: direction-only action and varying activation volume

We kept the action to the direction of activation on purpose, for three reasons.

First, this is where the operator's judgment lies. Before each quarter, the operator decides whether the imbalance calls for manual reserve and in which direction. The activated volume then depends on the requested demand and on the clearing of the European mFRR platform, which our data do not allow us to model.

Second, the data would not support learning the volume well. Only about 15% of quarters contain an activation, and their volumes vary widely. A continuous action learned from so few examples would add variance without making the evaluation clearer.

Third, a prescribed volume makes the comparison fairer. Every policy, including the operator's own decisions, is evaluated with the same volume rule, so differences come from when and in which direction a policy activates, not from how much. We tested 100 MW, 128 MW, 284 MW and a volume set from Elia's imbalance forecast, and the direction of every comparison holds (Table 17, Appendix A.11). Only at very large volumes do the differences shrink, because such large blocks overshoot.

The evaluation therefore covers the direction under a stated volume rule, not continuous dispatch. Learning the volume jointly with the direction is a natural extension, which we plan as future work.

Q1: multi-step prediction results

Yes. Table 14 and Figure 6 report multi-step validation. Given only the operator's actual actions, the grey-box evaluator runs freely for 96 quarters, a full day, and still reproduces the automatic reserve activation closely (R² = 0.968), including its extremes. The acceptance criteria were fixed before the first run, and the same criteria rejected the evaluator in two earlier market periods, which we therefore exclude (Appendix A.9, Table 14). The learned transition model of the initial submission is no longer used for outcomes, because it fails the control policies (see Q2).

Replay shows that the evaluator matches history, but much of this fit comes from the accounting itself. Whether it responds correctly to other actions is a separate question, which we answer under Q2.

Q2: accuracy for actions that differ from the historical decision

This is the key question, and replay alone cannot answer it. We use three complementary checks (see also our answer to W1 (a)).

First, control policies. Always holding, random actions and reversed operator actions are all ranked as worse than the operator (Appendix A.9, Figure 7), so the evaluator responds to actions in the right direction. The learned model of the initial submission failed this test.

Second, independent data. In Elia's minute-level data around almost 3,000 scheduled mFRR activations, the automatic reserves drop by 0.82–0.84 MW per MW of mFRR, close to the evaluator's 0.96 (Appendix A.10, Table 16). This supports the size of the response.

Third, the part no data can test. Whether the imbalance itself would have changed under another policy cannot be observed. We state this assumption and show that the conclusions hold unless market reactions almost fully cancel each activation (Appendix A.11, Figure 9).

The protocol's claims rest on the first two checks. The size of the outcome differences depends on the third, which Section 4 states.

AuthorsOpenReview ↗

Reply to Reviewer P6XY (Comment 3 of 3)

Q4: outcome metrics beyond mean |ACE|

We added them: high percentiles of |ACE|, the share of quarters outside Elia's control-error limits, and the activation of each reserve type (Section 3.3, Tables 9 and 11, Appendix A.8).

The ACE-based metrics are the same for every policy. In our evaluator this is largely by construction, because the automatic reserves absorb most of any change, so we draw no conclusion about ACE.

Reserve use is where the policies differ, and no single metric ranks them. The learned policies need less automatic reserve than the operator, but activate about twice as much manual reserve, and their total activation is higher. Whether this is better depends on how a TSO weighs these reserves against each other, and on cost and security margins, and this weighting is the TSO's decision. This extends the paper's argument: just as agreement does not reveal behavior, one outcome metric does not reveal the trade-off.

We did not estimate activation cost. Prices on the European platform respond to the activated volume, and the policies activate about twice the historical energy, so a cost estimate based on historical prices would be biased.

We would be glad to discuss any remaining questions.

Reviewer_JsPFOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether agreement with historical operator decisions reflects the behavior and effectiveness of learned power-balancing policies. Using Belgian grid data, it compares historical agreement, simulated intervention frequency, and simulated mean absolute Area Control Error (ACE). Reward changes, ambiguity-aware supervision, and selective deferral change policy behavior substantially while producing little change in mean |ACE|.

Critical strengths
  1. The paper correctly separates reproducing historical decisions from achieving desirable operational consequences. The reward intervention provides a concrete example where it shows that intervention rates change consistently across both algorithms and all five seeds, whereas changes in historical agreement are much less consistent. This is important for evaluating deployment behavior directly.
  2. The reward modification, targeted supervision, and decision-time deferral are complementary interventions. The presence of shared starting states, multiple training seeds, and explicit disclosure of the market-regime change improve transparency.
Critical weaknesses
  1. The high one-step prediction accuracy does not establish that the transition model captures the consequences of changing actions and the nearly unchanged ACE across substantially different policies could reflect limited simulator sensitivity rather than a meaningful operational finding.
  2. Every transition is treated as terminal which reduces the value target to immediate reward. Although acknowledged, this means the experiments evaluate contextual-bandit variants rather than multi-step offline RL. It is hard to be convinced that without a sequential-learning comparison, the evidence presented by the authors adequately support the paper’s broader RL framing.
Questions for authors
  1. What evidence validates the transition model’s predicted effect of alternative actions on ACE? Specifically, how do always-HOLD, random-action, and wrong-direction policies compare under this evaluator?
  2. Which state variables are available before the activation decision, specifically ACE and aFRR, and how are their timestamps aligned with the action labels?
2 comments
AuthorsOpenReview ↗

Reply to Reviewer JsPF (Comment 1 of 2)

We thank the reviewer for the careful review and for recognising the core of the paper: reproducing historical decisions and achieving good consequences must be evaluated separately. The reviewer's main concern, that our simulator might not respond to actions, was justified. We therefore built a grey-box evaluator, described in Section 2.3, validated it with the control policies the reviewer proposed (Appendix A.9, Figure 7), and checked it against minute-level data (Appendix A.10).

W1: one-step accuracy vs. consequences of changing actions; simulator sensitivity

We tested exactly this point. With the control policies the reviewer proposed in Q1, the learned transition model of the initial submission could not distinguish them from the operator (Appendix A.9, Figure 7), although it predicted historical transitions accurately (one-step R² = 0.97). The nearly unchanged ACE in the initial submission therefore came from the model, not from the policies.

The reason is physical. Most of the effect of an mFRR activation appears within the activation quarter. mFRR takes over the sustained part of the imbalance, and the faster automatic reserves are activated less. A model that predicts next-quarter ACE does not see this effect.

The revision therefore evaluates outcomes with a grey-box evaluator built on Elia's settlement accounting, in which an activation directly changes the imbalance left for the automatic reserves (Section 2.3). It passes three checks.

  • It reproduces full days of historical operation (Appendix A.9, Table 14, Figure 6).
  • It ranks every control policy as worse than the operator (Appendix A.9, Figure 7).
  • Its response to an activation is close to Elia's minute-level data, which do not use the fitted equation (Appendix A.10, Table 16).

Under this evaluator ACE still barely changes, because the automatic reserves absorb most of any change in mFRR (Appendix A.8, Table 11). The outcome is therefore measured where the action acts, in automatic reserve activation (Section 3.3, Figure 2). There, policies with similar agreement differ clearly, which strengthens the paper's central finding.

AuthorsOpenReview ↗

Reply to Reviewer JsPF (Comment 2 of 2)

W2: terminal transitions; contextual-bandit variants rather than multi-step offline RL

Section 2.2 states this explicitly. With terminal transitions, Stage 2 is offline contextual-bandit learning, and Q(s,a) estimates the expected immediate reward (Eqs. 6–7). This is a deliberate choice, and the work remains reinforcement learning for three reasons.

First, the policies are learned from rewards, not from labels, with offline RL algorithms (CQL, AWAC, CRR). Their conservative or advantage-weighted objectives matter even with one-step targets, because they keep the policy close to actions supported by the data. The contextual bandit is the one-step case of offline RL, not a different problem, and changing the reward changes the learned behavior consistently across methods and seeds (Figure 1).

Second, the paper is about what happens when such policies act in sequence. Each decision changes the state that the next decision sees, and this is exactly what agreement with historical decisions cannot capture. The closed-loop evaluation is therefore sequential, however the training targets were computed.

Third, one-step targets fit the physics of this task and do not drive our results. Most of the effect of an mFRR activation on the automatic reserves appears within the same quarter (Section 2.2, Appendix A.9). When we retrained CQL and AWAC with bootstrapped multi-step targets, decisions and outcomes barely changed (Appendix A.14).

Q1: always-HOLD, random-action and wrong-direction policies

We ran all three, plus three further control policies, under both evaluators on the same test days. The grey-box evaluator ranks every control as worse than the operator. Always holding raises automatic reserve activation slightly, reversed directions more, and random actions most. The learned transition model of the initial submission ranks all three as equal to the operator, which is why the revision evaluates outcomes with the grey-box evaluator. The results are in Appendix A.9 (Figure 7) and Table 15.

Q2: availability of state variables before the decision; timestamp alignment

Every feature uses only information published before the decision, 7.5 minutes before the quarter. ACE and aFRR enter from two quarters back, the last quarter published at that time. ENTSO-E measurements are lagged to respect their one-hour publication deadline. The forecast for the coming quarter is Elia's own published imbalance forecast. States and actions are aligned by timestamp, including in the operator baseline. Appendix A.7 (Table 8) gives the timing of every feature group.

On this state we retrained all methods and recomputed all results. Decision Agreement changes little, and the gap between agreement and behavior, the paper's central finding, holds (Table 1, Sections 3.2 to 3.5). We would be glad to discuss any remaining questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General comment to all reviewers and the area chair

We thank the three reviewers. In different ways, all three asked the same question: does our closed-loop evaluator actually respond to the actions it evaluates? The concern was justified, and it changed the paper.

We tested the learned transition model of the initial submission with simple control policies, such as always holding or reversing the operator's actions. It could not tell them apart from the operator, so its outcome comparisons could not separate the policies. We therefore built a grey-box evaluator from the TSO's settlement accounting, validated it against historical operation, and confirmed that it ranks the control policies as worse than the operator. Elia's minute-level data, analysed without this evaluator, support the size of its response.

With this evaluator, the paper's main message becomes clearer. Policies that agree with the operator on most decisions still behave very differently: they activate about twice as much manual reserve, which relieves the automatic reserves but increases total reserve activation. Agreement alone does not show this. The paper's original claim therefore holds: agreement does not reveal how a policy behaves in closed loop. With a validated evaluator, the claim becomes stronger, because agreement also misses the operational consequences of that behavior.

We also checked every state feature against its publication time, retrained all methods and report only current results. The research question, the three contributions and the structure of the paper are unchanged. The initial submission's findings on agreement and behavior are reproduced on the new state. What the revision changes is the outcome evaluation, which rests on a validated evaluator. Most of the new text is therefore in Section 2.3, the outcome results of Sections 3.2 to 3.5, the Discussion and the appendix. The list below shows where each change can be found.

What changed, and where to find it

Most changes follow from the main concern, whether the evaluator responds to actions:

  • Evaluator (Section 2.3, p. 5–6): outcomes now come from a grey-box evaluator built on Elia's settlement accounting. We kept the learned transition model as a reference, because its result on the control policies is what led to the change.
  • Validation (Appendix A.9, p. 19–22): the evaluator first replays full days of operation against criteria fixed in advance, and then has to rank the control policies as worse than the operator. The same criteria reject it before the MARI connection, so we use it only after.
  • Minute-level data (Appendix A.10, p. 22–23): an independent check of how strongly the automatic reserves respond to mFRR.
  • Outcome measure (Section 3.3, p. 8; Appendix A.8, p. 18–19): ACE hardly moves when the automatic reserves absorb most of an activation, so we now report automatic reserve activation. The ACE metrics are still in Table 11.

Other changes answer specific questions:

  • Causal state (Section 2.1, p. 3; Appendix A.7, p. 18): every feature is checked against its publication time, as P6XY asked. We retrained all methods, and agreement barely changed.
  • One-step targets (Section 2.2, p. 4): now stated explicitly, with the reason they fit this task (JsPF, P6XY).
  • Robustness (Appendix A.11, A.12 and A.14, p. 23–25): other activation volumes, how strongly the market would have to react to change the conclusions, relief per activated MWh, and multi-step training. The conclusions hold.
  • Germany (Appendix A.13, p. 25): the evaluator rebuilt for a second control area from public data (7ycB).
  • Related work (Section 1, p. 1–2): the four references 7ycB suggested.
  • Presentation (all sections): terms defined at first use, Section 2.3 explained step by step, and Figures 1 to 4 redrawn with current results (7ycB).
  • Code (Reproducibility Statement, p. 11): to be released upon publication.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.