- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper considers the problem of continual, safe reinforcement learning for rewiring networks (for example, infrastructure networks such as power grids). The argument behind the paper is that infrastructure networks experience different types of failures (e.g., failing components) at different stages and must keep functioning while keeping a reasonable cost of reconfiguring the network. While related works have studied constrained MDPs and continual RL in isolation, none have done so in the network rewiring setting. The paper loosely formulates the problem and proposes an RL approach for this setting. It is evaluated on synthetic networks with 50 nodes and mostly compares favourably with greedy approaches and vanilla RL, standalone safe RL, and standalone continuous RL.
Critical strengths
S1. The key idea behind the paper is interesting and novel, and identifies a reasonable challenge before reinforcement learning techniques for networks can be deployed to configure infrastructure networks in the real world.
Critical weaknesses
W1. Lack of clarity arising from LLM-driven writing
The key weakness of the work is its lack of clarity. The writing uses many specialist terms, concepts, and notations that are assumed to be known to the reader, but are often left undefined or assumed to be known throughout. To give examples:
- The paper never explicitly mentions what is the action space, only mentioning rewiring in a loose sense; but there are many ways to frame rewiring mathematically.
- The justification for combining the two RL techniques (end of Section 4) relies on continuous/safe RL notions that were never introduced.
- Similarly, the metrics (5.1) are hard to follow since relevant background is missing.
These are just a few examples, but the lack of clarity is a pervasive feeling that affects every paragraph of the text. The writing is locally coherent, but lacking a clear narrative that can be followed sentence by sentence. The general feeling reading the manuscript is that it is LLM-generated to a very large extent, and not human-originated writing that was merely edited using LLMs.
At least the authors do acknowledge in the AI use statement that LLMs carried out writing under supervision. However, is this sufficient to justify authorship? Given the pervasive lack of clarity in the paper, are the authors qualified to judge whether the work is technically correct? These are timely questions that extend beyond this paper.
W2. Limited evaluation scope and magic numbers
The method is only evaluated on synthetic graphs with 50 nodes that are generated using "classic" graph generative models with fixed parameters (L272-281) and under very specific parameter settings for the environment (L294-L302). There are many "magic parameters" that are stated but whose choice is never justified. T
he coverage of the experiments should be expanded substantially in order to establish whether the results hold in a general sense, or whether they are specific to the very narrow slice of (graph instance, environment parameters) combination that was tested. There are no experiments on real-world graphs.
The transferability of policies to different graphs, particularly to those larger in size, would support the motivation for the method. Is the method applicable out-of-the-box to graphs of different sizes?
W3. Limited novelty
The method combines two well-known techniques and thus methodological novelty is very limited. The paper argues that the application is more novel, but in my opinion existing applications of RL to infrastructure networks are somewhat under-stated. The lack of clarity and factful-but-limited interpretation of results also lead to little insight being gained from the paper in my opinion.
Questions for authors
Please address W1, W2, and W3 as these are the key points that affect my assessment. I am also listing some further queries and suggestions for improving the work below.
C1. The rewiring action space should be explicitly defined in the paper (see also W1).
C2. Regarding Q3, I would suggest citing the following two existing works since they deal with topological reconfiguration of infrastructure networks under robustness and efficiency objectives. The citations on L145 are overly generic, making the work appear more novel than it is in my opinion.
[1] Darvariu, V. A., Hailes, S., & Musolesi, M. (2021). Goal-directed graph construction using reinforcement learning. Proceedings: Mathematical, Physical and Engineering Sciences, 477(2254), 1-18.
[2] Yang, S., Kaili, M. A., Wang, B., Yu, T., & Zha, H. (2023). Learning to boost resilience of complex networks via neural edge rewiring. Transactions on Machine Learning Research.
C3. In my opinion, the 2x2 comparison should be explained much sooner than L234, since it is referenced several times before that, including in the list of contributions.
C4. Why are only some metrics bolded in Table 1 but not others? What is the argument for not bolding W and F_R?
C5. Section 5.2 is devoid of interpretation and does not explicitly say what the conclusions regarding the research questions were. It is not necessary to restate results that are already visible in the figures, or redefine experimental protocol / definition of metrics. At least part of this section should be dedicated to drawing conclusions about the questions (currently left to the final section, too late). This is another hallmark of AI writing.