- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper extends RL-based optimal taxation frameworks (such as TaxAI) from a single-group setting to a multi-group setting. Existing works model the interaction between taxpayers and a government as a single, isolated economy, overlooking inter-jurisdictional fiscal competition where independent governments simultaneously compete for mobile capital. Combining an intra-group Stackelberg leader–follower game with an inter-group non-cooperative tax game, the paper formulates the Multi-Group Hierarchical Tax Game (MHTG). It then solves this problem using Multi-Group PPO (MGPPO), which integrates dual intra-/inter-group critics, two-timescale sampling, and a capital-mobility curriculum. Evaluated over a 300-year simulation horizon, MGPPO improves economic survival and reduces income and wealth inequality (Gini coefficients) compared to IPPO, MAPPO, and MAA2C, while improving mean GDP in the two-group setting.
Critical strengths
- Since capital in modern age is often mobile, extending RL-based optimzation for taxation policy from a close single-government economies to multijurisdiction setup is an reasonable and straightforward extension
- The authors incorporates various economic principles and theory to ground their simulation.
- The designs of the mechanism is straightfoward and reasonable.
Critical weaknesses
- The baseline comparisons were limited to flat policies. A comparison with more advanced and adaptive methods, such as TaxAI would better highlight the advantages.
- Since the paper is being published to an CS venue, it would be better to explain which of the ideas are ideas proposed by the authors (whether based on educated guesses or with proven empirical/theoretical studies). For example, why is the example horizon set to 300 years? 300 years is a very long time and given the political / technological / socioeconomic changes that can occur within that the such a long time frame (that's the time frame of industrial revolution to present day), it would be difficult to simulate an economy based on taxation alone.
- From the point of view as a new RL algorithm, many of the concepts the MGPPO uses feels like it has been explored before, and MGPPO is combining them to a new problem. It would be better to see some ablation studies to highlight how the MGGPO works, why it is novel, and how the specific combination of multi-agent features MGPPO makes it better suited to solve this problem, better than how the existing algorithms would fare in such a benchmark.
Questions for authors
I would request
- Additional baselines to compare the performance of state-of-the-art approaches and similar approaches against what MGPPO is proposing.
- Further justification on experiment setup, including the long time horizon.
- Ablation Studies to highlight the impact and elaborate the MGPPO's choice of combining RL concepts to solve the multijurisdictional taxation problem.
3 comments
Thank you for your response!
Thank you for your response! The explanations cleared many of my questions, and I think including the comparison BMFAC certainly made this paper much stronger. Therefore, I have raised the score by 1 point. Thank you for your hard work.
The reason why I am not raising the score by 2 points instead of 1 is that I cannot fully verify novelty of the work. The components of this algorithm already existed which undermines some novelty. However combining them in the manner the authors suggested certainly had some new merits as shown in the results. The empirical results seems to be strong. Therefore, while I am only increasing the score by one point, I am also OK with this paper being accepted if the other reviewers and area chair finds the algorithm to be novel enough to be accepted to the venue.
Response to the Reviewer's Follow-up Comments
Thank you for your thoughtful reassessment and for raising the score. We are pleased that our explanations and the BMFAC comparison helped address your concerns, and we appreciate your recognition of the empirical results.
We agree that the individual learning components build on established methods. We would like to clarify three aspects of our contribution.
-
A hierarchical multi-group tax game environment. We extend the conventional single-group taxation setting by introducing inter-group capital mobility. The resulting hierarchical multi-agent simulation environment captures government–household interactions within each group and competition among governments. These coupled interactions pose additional challenges for policy optimization and motivate the design of MGPPO.
-
A learning architecture tailored to this game structure. As you noted, we combine the joint government critic, heterogeneous decision sampling, and curriculum learning to address these challenges. We regard their adaptation and integration for bilevel multi-group games as one contribution, while acknowledging that the individual techniques have precedents.
-
Information structuring. We would also like to clarify that we structure the information supplied to each agent’s critic. In the bilevel, multi-group architecture, the influence of different agents’ actions is asymmetric. As discussed in our earlier response, the policy of the government in Group 1 may be strongly affected by the actions of households within its own group and by the policies of other governments, while being only weakly affected by household actions in other groups. Based on this observation, we structure each critic’s inputs around the relevant interactions, reducing unnecessary information. This is also the main motivation for introducing the joint government critic. To our knowledge, this specific information-structuring problem in bilevel, multi-group tax games has not been explicitly addressed in prior work.
In our baseline comparisons, existing MARL methods, such as MAPPO and IPPO, cannot be directly applied to the proposed hierarchical multi-group environment without additional adaptations, as detailed in Section 5.1 (P8 L381) of our paper. To ensure fair comparisons, all methods use the same heterogeneous decision sampling and curriculum settings, controlling for these training mechanisms so that the comparison primarily reflects differences in information structure and critic architecture. MGPPO achieves higher rewards than the adapted MAPPO and IPPO baselines, while the corresponding ablation further evaluates the contribution of the joint government critic. These results support the effectiveness of the proposed design in our environment.
We hope these explanations clarify our contribution and its relationship to existing methods. If you have any further concerns, please do not hesitate to let us know. We sincerely hope that our clarifications can address your concerns and earn your confidence and support.
Authors' Rebuttal
Concern 1: Additional baselines to compare the performance of state-of-the-art approaches and similar approaches against what MGPPO is proposing.? (Weakness 1; Question 1)
A1: We thank the reviewer for this important suggestion. Methods that explicitly consider government–household interactions can provide a more relevant comparison than flat MARL baselines. As there is currently no established baseline designed specifically for a multi-group tax-policy environment, we adapted BMFAC, which was developed in TaxAI for single-group government–household games, and compared it with MGPPO. In our adaptation, each government learns to respond to inter-group competition using its own information. For details, please refer to the blue-highlighted revisions in Section 5.1: economic outcomes (P8, L 391), reward analysis (P8, L 424), and action analysis (P8, L436).
For N=2, BMFAC yields relatively low government and household rewards. For N=5, household rewards are high but fluctuate, while government rewards remain relatively low. Its single-group design and reliance on individual government information may limit its ability to capture inter-group competition. Historical samples used in off-policy updates may also become less representative as agents’ policies evolve, making learning more difficult.
Concern 2: Which ideas are proposed by the authors, and why is the simulation horizon set to 300 years despite possible political, technological, and socioeconomic changes? (Weakness 2; Question 2)
A2: Thank you for requesting clarification on our contributions. We extend the single-group tax game to a multi-group setting by introducing an inter-group capital-mobility mechanism and constructing the corresponding simulation environment. Furthermore, we develop MGPPO to solve the resulting bilevel multi-group learning problem.
We use episode survival length to assess the simulated economy’s ability to remain operational. Additionally, as shown in Appendix~E, the action trajectories become relatively stable after approximately 150--200 steps. A 300-step horizon therefore covers both the initial adjustment phase and a subsequent period for assessing whether this behavior persists. Importantly, an environment step does not strictly correspond to one calendar year in the real world, as tax policies are not necessarily adjusted annually. The simulation is intended for stylized policy analysis rather than a literal 300-year economic forecast.
Concern 3: What is the methodological contribution of MGPPO? (Weakness 3)
A3: Thank you for requesting clarification. MGPPO’s methodological contribution is a hybrid explicit and implicit modeling strategy tailored to the bilevel, multi-group game structure.
In multi-agent reinforcement learning, incorporating information about other agents can improve coordination and training stability, such as MAPPO. However, conditioning policies or critics on overly large joint inputs may increase optimization difficulty and impair performance. In our proposed bi-level, multi-group architecture, the influence among agents is inherently asymmetric. For example, the policy of the government in Group 1 may be strongly affected by the actions of households within its own group and by the policies of other governments, while being only weakly affected by household actions in other groups. To balance the benefits of coordination against the complexity of modeling joint information, MGPPO structures its learning signals around inter- and intra-group interactions. At the algorithmic level, it combines explicit and implicit modeling to provide each actor and critic with only the action information relevant to its interactions, while the joint government critic integrates intra- and inter-group evaluations. At the training level, hierarchical decision sampling and curriculum learning are introduced to accommodate the different decision timescales and increasing interaction complexity of the bilevel multi-group tax game.
For details, please refer to Section 4.2 (P6, L280).
Concern 4: Can ablations justify its combination of existing MARL concepts? ( Question 3)
A4: For both N=2 and N=5, we evaluate four model variants: M1 is the complete MGPPO model; M2 replaces the joint government-critic architecture with a unified critic that processes all available inputs; M3 removes hierarchical decision sampling; and M4 removes curriculum learning.
Figure 3 shows that removing hierarchical decision sampling substantially reduces both government and household rewards. For N=2, M3 performs considerably worse, while M1, M2, and M4 achieve similar results. For N=5, the full model attains higher government rewards in the later stages of training. These results support the effectiveness of the proposed components under the evaluated settings.
For details, please refer to Section 5.2 (P9, L459), Figure 3 (P10).