- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The problem addressed in this paper is power grid planning. The approach leverages LLMs to generate candidate updates which contain instantiated actions. These updates are structured into three categories, each focused on a different criterion (such as diagnostic evidence, investment cost and security requirements). A heuristic search is used to find an improved plan with no optimality guarantees. An empirical evaluation is presented where a few variations of the system at hand are compared against each other (full version versus partial versions), with no external baseline included.
Critical strengths
The problem addressed is important. Leveraging LLMs is an interesting and worth-of-being-study approach.
Critical weaknesses
No baseline approaches are included in the evaluation. This makes it difficult if possible at all to understand how the SCOPE approach compares to the state of the art. The presentation of the technical approach in the main paper is not sufficiently specific. An example and more details on how each step works would help. Only one grid topology, namely IEEE 118 is used in experiments. More topologies would add breadth and possibly new insights not visible in the one topology considered.
Questions for authors
What baseline methods would be appropriate for a comparison?
2 comments
Follow Up
Thank you once again for your thoughtful feedback. We appreciate the time you have dedicated to reviewing our work.
To summarize our rebuttal:
- W1 & Q1: We added two non-LLM baselines, VeraGrid’s genetic algorithm and particle swarm optimization, using the same candidate actions, costs, and physical verifier as SCOPE. Across three runs on all 100 scenarios, all methods achieve 100% secure completion. SCOPE achieves approximately 20% lower investment cost, with somewhat longer runtime (11.5 vs. 9.1 minutes per scenario).
- W2: We added a worked example of one complete planning round to make the technical approach more concrete.
- W3: We clarified that the current evaluation covers one topology, the IEEE 118-bus system, and discussed evaluation on additional power systems as future work.
As the discussion phase is drawing to a close, we would be grateful if you could revisit our responses. Your feedback on whether these additions address your concerns would be very valuable to us. If you feel that our responses have adequately addressed your concerns, we would appreciate it if you could consider updating your scores accordingly.
Thank you for your time and consideration.
Response to Reviewer RUB2
We thank the reviewer for the review and for recognizing the importance of the problem and the interest of an LLM-based approach. We address each weakness and the question below.
Weaknesses
W1. No baseline approaches.
We added two established, non-LLM planning baselines that use the same candidate actions, costs and pandapower verifier: VeraGrid, which searches candidate investments with a genetic algorithm, and particle swarm optimization (PSO). Over three seeds on all 100 scenarios, both secure every scenario, at a mean investment cost of USD 60.80M (VeraGrid) and USD 60.90M (PSO) (non-LLM baseline comparison table). SCOPE also secures every scenario in all three runs, at a mean investment cost of USD 48.76M ± 0.12, about 20% lower than both baselines; its run time is somewhat longer (11.5 vs. 9.1 minutes per scenario), as it includes LLM inference.
W2. The technical approach is not sufficiently specific.
We added to the revised manuscript a worked example of one complete planning round.
W3. Only one grid topology.
The current study focuses on the planning method and evaluates it on one topology, the IEEE 118-bus system, which is large enough that security-constrained reinforcement is non-trivial while every candidate can be verified with full AC power flow and N-1 analysis. We now state this scope explicitly in the revised manuscript, and discuss extending SCOPE to further power systems as future work.
Question
Q1. What baseline methods would be appropriate for a comparison?
Appropriate baselines include security-constrained expansion planning with mathematical optimization, metaheuristics such as genetic algorithms and particle swarm optimization, and simple heuristics based on the verifier's diagnostics. We added VeraGrid (genetic algorithm) and particle swarm optimization, both using the same pandapower verifier as SCOPE (non-LLM baseline comparison table).