- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses the problem of multi-agent coordination of power grids under partial observability and cross regional coupling. It proposes GRIDREFINE, a decentralized topology-control framework for power grids in which each regional agent is powered by an LLM. In particular, GRIDREFINE's regional agents use historical outcomes and local physical calculations to select topology actions, coordinating when neighbouring proposals interact.
The framework is evaluated on Grid2OP 14-bus grid and a 36-bus grid (with and without planned maintenance), against Do-Nothing, MAPPO and a non-LLM retrieval controller. Three LLM backbones are tested: Gemma (4.5B), Qwen (27B) and DeepSeek (284B MoE). The results are compared to the MAPPO baseline (Yu et al., 2022) and demonstrate that QWEN and DeepSeek overcome MAPPO by 28–42 pp in mean survival.
The study report some ablations and detailed appendix to support their claims.
Critical strengths
- Well motivated mechanisms through examples; for example in Fig. 2, where a historical failure overrides favourable DC estimates, is convincing.
- Extensive appendices with sufficient reproducibility materials and transparent discussions of limitations and failures (of Gemma).
- Insightful ablations and negative results, with sufficient discussions.
Critical weaknesses
The system is carefully engineered and thoroughly documented. The central empirical claims, however, rest on a weak, apparently unconverged baseline, missing natural baselines, and ablations with one seed and 20 episodes. The coordination mechanism is not convincingly shown to be the source of the gain.
-
The choice of MAPPO as sole baseline, and unfair comparison of MAPO. Figure 4 in appendix clearly shows MAPPO is not converged and can achieve significantly better results. Currently, MAPPO is below Do-Nothing on both Bus36 settings (44.7 vs 67.7; 27.6 vs 44.0), which is absurd, and confirm it was not properly/sufficiently trained.
-
The results are not significant:
-- Relative to Do Nothing, Qwen on Bus36 gains about 10.7 pp, totally within the standard deviation margins Qwen (61.4 ± 12.4) and Do-Nothing (44.0 ± 16.9).
-- Table 2 and Fig. 3 use 20 validation episodes and one seed (seed 0), no confidence intervals, and cumulative (not leave-one-out) ablations.
-- other qualitative claims.
-
Contradicting findings between main and appendix: Appendix D.3 reports that, over 7,788 coordinated events, there were 75 revisions, zero shared-line objections, zero fallbacks, and 1.01 average review cycles. Which means there is no benefit to the coordination; while the main ablation in Table 2 attributes +14.4 pp (Bus14) to coordination.
-
No demonstration of the generalization capabilities: All memory is grid-specific and built from the same data distribution (training chronics of the same grid). The test-time benefit of retrieval from 700/400 chronics may largely reflect closeness of test states to stored states on small benchmarks.
Questions for authors
-
Provide more up to date baselines: Including best single agent approaches (L2RPN winning agents) or more recent approaches like PowRL (https://arxiv.org/html/2212.02397v2), or decentralized coordination in power systems. Distributed/decentralized optimization for multi-area TSOs (ADMM, auxiliary problem principle, DCOP; see surveys such as Kargarian et al., 2016) is the established way to coordinate across control areas under partial information. The paper’s claim that “existing approaches lack a mechanism” for coordination ignores this literature. At least a discussion, and ideally a rule-based negotiation baseline, is needed.
-
Please validate the statistical significance of the claims, especially the qualitative ones (“coordination also improves survival with planned line disconnections”, “mining improves in all three settings”)
-
Explain the contradiction about the benefits of coordination.
-
Provide actual generalization results; for example: transfer to unseen topologies or grids; distribution shift (e.g., different seasonal profile, renewable penetration, enabled opponent/outages); robustness to memory size.
-
What is the test-time survival of the greedy AC-lookahead controller (with and without the one-region-per-step rule) on the same splits and catalogs? How many AC simulator calls does it use per decision compared with GRIDREFINE?