- Rating
- 2(−1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies symbolic interpretation of multi-agent reinforcement learning (MARL) policies in the context of active voltage control on the MAPDN case33 distribution network with six photovoltaic (PV) inverter agents trained with MATD3.
Critical strengths
(1) Well-motivated and coherent conceptual framing. (2) Cross-policy structural reuse is a genuinely interesting result. (3) Closed-loop joint replacement is a strong evaluation.
Critical weaknesses
(1) Evidence is limited to a single environment, a single algorithm, and a single teacher. All conclusions rest on MAPDN case33 with six MATD3 agents. The paper's framing ("multi-agent reinforcement learning policies" generally) is considerably broader than the evidence. Nothing is known about whether structural multiplicity, MDL depth selection, or cross-policy reuse behaves similarly on larger networks (e.g., case123), other MARL algorithms (IPPO, MADDPG, QMIX), or non-power-systems tasks. (2) No comparison against alternative interpretable-policy baselines. The paper never compares against the natural competitors.
Questions for authors
(1) What is the end-to-end computational cost (CPU-hours, hardware) of Stage I + Stage II + selection + closed-loop evaluation, and how does it scale with the number of agents?
Response to Reviewer nNRk: Expanded Evaluation and Computational Cost
Thank you for recognizing the paper's conceptual framing, cross-policy structural reuse, and joint closed-loop evaluation. The revised manuscript evaluates these contributions across two MARL algorithms, two network scales, independently trained teachers, and six interpretable-policy baselines, alongside a computational-cost analysis. All changes and additions are highlighted in blue in the revised manuscript. Unless otherwise indicated, line numbers refer to the revision.
1. Experimental breadth and generalizability (Critical Weakness 1). Building on the original four-target structural reuse experiment (original Sec. 4.3 and App. B.4), PRSI is now evaluated on 33-bus MATD3, 33-bus HASAC, and 141-bus MATD3, extending joint symbolic control from 6 to 22 inverters. Across all three settings, residual refinement lowers within-inverter mean held-out action NRMSE relative to Stage I for every inverter (Fig. 3, lines 421–428). MDL adapts explanation depth path by path, selecting residual corrections on 20/24, 21/25, and 78/105 paths, respectively, while keeping the compact Stage-I formulas on the remaining 35 paths, where forced refinement adds a median of 3 symbolic search-complexity units (Table 23, lines 430–437, 2190–2199). Fixed symbolic structures also transfer across four independently trained target teachers in each setting, with coefficient refitting improving held-out fidelity over direct source-coefficient reuse at the largest calibration budget. Under joint nominal MATD3 deployment, PRSI's voltage-violation-rate difference from the neural teachers stays within 0.087 percentage points on 33 buses and 0.837 on 141 buses (lines 414–418). These results demonstrate consistent residual refinement, adaptive explanation depth, and cross-teacher structural reuse across different MARL algorithms and network sizes.
2. Comparison with interpretable policies (Critical Weakness 2). We compare PRSI with six baselines—direct PySR, OLS, Lasso, CART, DAgger+CART, and VIPER—on held-out fidelity, model size, and joint closed-loop control across nine matched operating conditions (Table 1, Tables 19–20). PRSI achieves the lowest nine-condition mean voltage-violation rate among all seven methods on both MATD3 networks. On 141 buses, it achieves 4.383%, a 24.2% reduction relative to direct PySR (5.781%; paired reduction 1.399 percentage points, 95% CI [1.198, 1.600]), and below the next-best baseline, CART (5.188%). Under the two conditions combining reduced PV generation with elevated demand, the mean reduction relative to direct PySR reaches 4.546 percentage points on 141 buses and 0.273 on 33 buses (lines 408–412). On 33-bus MATD3, PRSI achieves 1.596% (paired reduction against PySR 0.090 percentage points, 95% CI [0.057, 0.122]); on 33-bus HASAC, it reduces direct PySR's violation rate by 43.6% (2.909% to 1.641%). PRSI also delivers more compact symbolic rules: within-inverter mean symbolic search complexity is lower than DSR across all three settings (Table 1, Panel B), with a 34.1–63.0% reduction at comparable cumulative search time on 33-bus MATD3 (lines 439–444). Together, these results demonstrate compact symbolic interpretation with effective joint regulation in physically coupled multi-agent systems.
3. Computational cost and hardware (Question 1). Appendix D.4, Table 24 quantifies Stage-I plus Stage-II search costs (lines 2200–2225). At 1,600 iterations per search and one search seed, estimated per-inverter costs are 104.007 min (33-bus MATD3), 107.738 min (33-bus HASAC), and 118.934 min (141-bus MATD3). Across the two MATD3 workloads, the inverter count increases 3.7-fold (6 to 22), while per-inverter search time increases by 14.4%, with aggregate search workloads of 10.40 and 43.61 hours, respectively. The recorded 141-bus Stage-I execution completed 198 searches across three seeds in 4.57 wall-hours using six workers (27.32 summed task-hours). Representative-path selection was performed on a single MacBook Pro with 32 GiB unified memory, using four worker processes and one BLAS thread per process. The 141-bus closed-loop evaluation covers nine conditions × 100 paired episode seeds × 239 control steps (lines 2106–2115). Complete process CPU times for the full pipeline were not separately recorded; we report the search-time estimates and measured 141-bus Stage-I wall time.