- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces a meta-RL solution for config adaptation in hierarchical cooperative MARL tasks. By experimenting on a traffic signal control task, the proposed solution shows better performance than conventional hierarchical MARL solutions.
Critical strengths
-
The performance of the proposed method is good, outperforming previous solutions significantly.
-
The formulation of the meta-MDP is good and practically useful. The proposed bilevel optimization addresses the MDP well.
Critical weaknesses
-
The motivation of this paper is unclear, especially regarding the three-level design. An ablation study to show the efficacy of each level is necessary.
-
The experiment is not sufficient. Comparing with mature baselines for the signal control task is helpful. Moreover, the experiment scope is too narrow, including only one task.
-
The writing should be improved. For example, the notions in the introduction are not defined when they are first used.
Questions for authors
-
What is the benefit of the proposed three-level hierarchies to conventional two-level solutions and the role-based MARL methods, where the role assignment mechanism is very similar?
-
Can the proposed method be used in other hierarchical RL methods, or is it limited to the proposed three-level framework?