acceptodds
Under review as a conference paper at ICLR 2027

Hierarchical Multi-Agent Reinforcement Learning for Multi-Group Tax Game

Abstract

Taxation is a fundamental instrument of economic policy and has long been studied through economic models. However, most existing taxation models focus on interactions within a single economic group, typically with one government and multiple households. Such a setting overlooks interactions among independent economic groups, where governments may compete for economic resources through fiscal policies. To capture these interactions, we formulate taxation as a hierarchical multi-group tax game. Within each group, a government sets tax policies while households respond, forming a hierarchical government–household interaction. Across groups, governments compete through fiscal policies, coupling the economic dynamics of distinct groups. This coupled structure poses challenges for standard multi-agent reinforcement learning (MARL) methods. To this end, we propose Multi-Group PPO (MGPPO), a bilevel MARL algorithm designed for hierarchical multi-group interactions. MGPPO incorporates Hierarchical Sampling to coordinate learning across agent levels and Curriculum Learning to improve training stability. We further develop a multi-group taxation simulation environment grounded in classical economic models, supporting the evaluation of fiscal policies under inter-group competition. Simulations show that in the two-group settings, MGPPO reduces the income and wealth Gini coefficients by 5.0% and 4.9%, respectively, and increases mean GDP by 4.1% compared with IPPO.

Primary area:
reinforcement learning
Keywords:
Multi-Agent Reinforcement Learning, Economic Simulation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_A6feOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper extends RL-based optimal taxation frameworks (such as TaxAI) from a single-group setting to a multi-group setting. Existing works model the interaction between taxpayers and a government as a single, isolated economy, overlooking inter-jurisdictional fiscal competition where independent governments simultaneously compete for mobile capital. Combining an intra-group Stackelberg leader–follower game with an inter-group non-cooperative tax game, the paper formulates the Multi-Group Hierarchical Tax Game (MHTG). It then solves this problem using Multi-Group PPO (MGPPO), which integrates dual intra-/inter-group critics, two-timescale sampling, and a capital-mobility curriculum. Evaluated over a 300-year simulation horizon, MGPPO improves economic survival and reduces income and wealth inequality (Gini coefficients) compared to IPPO, MAPPO, and MAA2C, while improving mean GDP in the two-group setting.

Critical strengths
  • Since capital in modern age is often mobile, extending RL-based optimzation for taxation policy from a close single-government economies to multijurisdiction setup is an reasonable and straightforward extension
  • The authors incorporates various economic principles and theory to ground their simulation.
  • The designs of the mechanism is straightfoward and reasonable.
Critical weaknesses
  • The baseline comparisons were limited to flat policies. A comparison with more advanced and adaptive methods, such as TaxAI would better highlight the advantages.
  • Since the paper is being published to an CS venue, it would be better to explain which of the ideas are ideas proposed by the authors (whether based on educated guesses or with proven empirical/theoretical studies). For example, why is the example horizon set to 300 years? 300 years is a very long time and given the political / technological / socioeconomic changes that can occur within that the such a long time frame (that's the time frame of industrial revolution to present day), it would be difficult to simulate an economy based on taxation alone.
  • From the point of view as a new RL algorithm, many of the concepts the MGPPO uses feels like it has been explored before, and MGPPO is combining them to a new problem. It would be better to see some ablation studies to highlight how the MGGPO works, why it is novel, and how the specific combination of multi-agent features MGPPO makes it better suited to solve this problem, better than how the existing algorithms would fare in such a benchmark.
Questions for authors

I would request

  • Additional baselines to compare the performance of state-of-the-art approaches and similar approaches against what MGPPO is proposing.
  • Further justification on experiment setup, including the long time horizon.
  • Ablation Studies to highlight the impact and elaborate the MGPPO's choice of combining RL concepts to solve the multijurisdictional taxation problem.
3 comments
AuthorsOpenReview ↗

Authors' Rebuttal

Concern 1: Additional baselines to compare the performance of state-of-the-art approaches and similar approaches against what MGPPO is proposing.? (Weakness 1; Question 1)

A1: We thank the reviewer for this important suggestion. Methods that explicitly consider government–household interactions can provide a more relevant comparison than flat MARL baselines. As there is currently no established baseline designed specifically for a multi-group tax-policy environment, we adapted BMFAC, which was developed in TaxAI for single-group government–household games, and compared it with MGPPO. In our adaptation, each government learns to respond to inter-group competition using its own information. For details, please refer to the blue-highlighted revisions in Section 5.1: economic outcomes (P8, L 391), reward analysis (P8, L 424), and action analysis (P8, L436).

For N=2N=2, BMFAC yields relatively low government and household rewards. For N=5N=5, household rewards are high but fluctuate, while government rewards remain relatively low. Its single-group design and reliance on individual government information may limit its ability to capture inter-group competition. Historical samples used in off-policy updates may also become less representative as agents’ policies evolve, making learning more difficult.


Concern 2: Which ideas are proposed by the authors, and why is the simulation horizon set to 300 years despite possible political, technological, and socioeconomic changes? (Weakness 2; Question 2)

A2: Thank you for requesting clarification on our contributions. We extend the single-group tax game to a multi-group setting by introducing an inter-group capital-mobility mechanism and constructing the corresponding simulation environment. Furthermore, we develop MGPPO to solve the resulting bilevel multi-group learning problem.

We use episode survival length to assess the simulated economy’s ability to remain operational. Additionally, as shown in Appendix~E, the action trajectories become relatively stable after approximately 150--200 steps. A 300-step horizon therefore covers both the initial adjustment phase and a subsequent period for assessing whether this behavior persists. Importantly, an environment step does not strictly correspond to one calendar year in the real world, as tax policies are not necessarily adjusted annually. The simulation is intended for stylized policy analysis rather than a literal 300-year economic forecast.


Concern 3: What is the methodological contribution of MGPPO? (Weakness 3)

A3: Thank you for requesting clarification. MGPPO’s methodological contribution is a hybrid explicit and implicit modeling strategy tailored to the bilevel, multi-group game structure.

In multi-agent reinforcement learning, incorporating information about other agents can improve coordination and training stability, such as MAPPO. However, conditioning policies or critics on overly large joint inputs may increase optimization difficulty and impair performance. In our proposed bi-level, multi-group architecture, the influence among agents is inherently asymmetric. For example, the policy of the government in Group 1 may be strongly affected by the actions of households within its own group and by the policies of other governments, while being only weakly affected by household actions in other groups. To balance the benefits of coordination against the complexity of modeling joint information, MGPPO structures its learning signals around inter- and intra-group interactions. At the algorithmic level, it combines explicit and implicit modeling to provide each actor and critic with only the action information relevant to its interactions, while the joint government critic integrates intra- and inter-group evaluations. At the training level, hierarchical decision sampling and curriculum learning are introduced to accommodate the different decision timescales and increasing interaction complexity of the bilevel multi-group tax game.

For details, please refer to Section 4.2 (P6, L280).

Concern 4: Can ablations justify its combination of existing MARL concepts? ( Question 3)

A4: For both N=2N=2 and N=5N=5, we evaluate four model variants: M1 is the complete MGPPO model; M2 replaces the joint government-critic architecture with a unified critic that processes all available inputs; M3 removes hierarchical decision sampling; and M4 removes curriculum learning.

Figure 3 shows that removing hierarchical decision sampling substantially reduces both government and household rewards. For N=2N=2, M3 performs considerably worse, while M1, M2, and M4 achieve similar results. For N=5N=5, the full model attains higher government rewards in the later stages of training. These results support the effectiveness of the proposed components under the evaluated settings.

For details, please refer to Section 5.2 (P9, L459), Figure 3 (P10).

Reviewer_A6feOpenReview ↗

Thank you for your response!

Thank you for your response! The explanations cleared many of my questions, and I think including the comparison BMFAC certainly made this paper much stronger. Therefore, I have raised the score by 1 point. Thank you for your hard work.

The reason why I am not raising the score by 2 points instead of 1 is that I cannot fully verify novelty of the work. The components of this algorithm already existed which undermines some novelty. However combining them in the manner the authors suggested certainly had some new merits as shown in the results. The empirical results seems to be strong. Therefore, while I am only increasing the score by one point, I am also OK with this paper being accepted if the other reviewers and area chair finds the algorithm to be novel enough to be accepted to the venue.

AuthorsOpenReview ↗

Response to the Reviewer's Follow-up Comments

Thank you for your thoughtful reassessment and for raising the score. We are pleased that our explanations and the BMFAC comparison helped address your concerns, and we appreciate your recognition of the empirical results.

We agree that the individual learning components build on established methods. We would like to clarify three aspects of our contribution.

  1. A hierarchical multi-group tax game environment. We extend the conventional single-group taxation setting by introducing inter-group capital mobility. The resulting hierarchical multi-agent simulation environment captures government–household interactions within each group and competition among governments. These coupled interactions pose additional challenges for policy optimization and motivate the design of MGPPO.

  2. A learning architecture tailored to this game structure. As you noted, we combine the joint government critic, heterogeneous decision sampling, and curriculum learning to address these challenges. We regard their adaptation and integration for bilevel multi-group games as one contribution, while acknowledging that the individual techniques have precedents.

  3. Information structuring. We would also like to clarify that we structure the information supplied to each agent’s critic. In the bilevel, multi-group architecture, the influence of different agents’ actions is asymmetric. As discussed in our earlier response, the policy of the government in Group 1 may be strongly affected by the actions of households within its own group and by the policies of other governments, while being only weakly affected by household actions in other groups. Based on this observation, we structure each critic’s inputs around the relevant interactions, reducing unnecessary information. This is also the main motivation for introducing the joint government critic. To our knowledge, this specific information-structuring problem in bilevel, multi-group tax games has not been explicitly addressed in prior work.

In our baseline comparisons, existing MARL methods, such as MAPPO and IPPO, cannot be directly applied to the proposed hierarchical multi-group environment without additional adaptations, as detailed in Section 5.1 (P8 L381) of our paper. To ensure fair comparisons, all methods use the same heterogeneous decision sampling and curriculum settings, controlling for these training mechanisms so that the comparison primarily reflects differences in information structure and critic architecture. MGPPO achieves higher rewards than the adapted MAPPO and IPPO baselines, while the corresponding ablation further evaluates the contribution of the joint government critic. These results support the effectiveness of the proposed design in our environment.

We hope these explanations clarify our contribution and its relationship to existing methods. If you have any further concerns, please do not hesitate to let us know. We sincerely hope that our clarifications can address your concerns and earn your confidence and support.

Reviewer_ZiXnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This work introduces a new multi-agent reinforcement learning algorithm for taxation modelling within and between economic groups (nations/governments). The main contribution of this work is to propose a model that includes both intra- and inter-group interactions in the simulation. This is done by separating the training mechanism into two levels: intra- and inter-group training, updated separately at different intervals. The resulting algorithm is evaluated experimentally in comparison with other state-of-the-art MARL algorithms and shown to achieve higher economic markers.

Critical strengths
  • Algorithmic and modelling choices are described clearly.
  • This work appears to address a real gap in the relevant literature by extending the modelling framework to encompass nation-level interactions.
  • The modelling of economic factors appears to have a solid theoretical foundation in the domain-specific literature.
  • statistically solid experimental design: reported results are derived from 20 different runs with varying random seeds.
Critical weaknesses
  • In the theoretical description, too much of the underlying detail is outsourced to the appendix. Section 3 repeatedly refers to equations and variables that have not been introduced in the main section. While it is reasonable to shift the underlying details to the appendix, the broad strokes of the contribution should nevertheless be understandable from the main section.

  • While the analysis of scalability is welcome, it does not seem to demonstrate convergence, performance, or survival of the system or compare to the performance of the other state-of-the-art algorithms. Only the change in parameter results is discussed.

Questions for authors

Could the authors clarify how hyperparameters were selected for the experiments? This includes the tuning of the compared algorithm, and the relevant parameters for the proposed algorithm (e.g. government vs. household decision period; curriculum learning rate, etc.)

4 comments
AuthorsOpenReview ↗

Authors' Rebuttal

We thank the reviewer for recognizing the scalability analysis and for identifying areas where the presentation and evaluation needed clarification. We have revised the manuscript as summarized below.


Concern 1: Can the authors provide evidence of convergence? (Weaknesses 2)

A1: Thank you for highlighting the need for convergence evidence. We have added government and household reward trajectories for N=2,5,10,20,N=2,5,10,20, and 5050.

For details, please refer to Section 5.1, Figure 2 (P 9, L 414) and Appendix D4, Figure 7 (P20, L1026).


Concern 2: Compare MGPPO with state-of-the-art algorithms. (Weaknesses 2)

A2: Thank you for suggesting a stronger baseline comparison. We adapted BMFAC, a state-of-the-art method from TaxAI for single-group government–household interactions, to our multi-group environment. Each government models inter-group competition using its individual information. We compare BMFAC with MGPPO, IPPO, MAPPO, and MAA2C in terms of economic outcomes, rewards, and actions. For details, please refer to the blue-highlighted revisions in Section 5.1: economic outcomes (P. 8, L 391), reward analysis (P 8, L 424), and action analysis (P8, L436).


Concern 3: How were the experimental hyperparameters selected, including baseline tuning, decision periods, and the curriculum schedule? (Question 2)

A3: Thank you for requesting clarification on the hyperparameter selection.

Baseline settings and tuning. The main training hyperparameters are summarized in **Appendix C2, Tables 3–5 (P15, L786). ** Additionally all algorithms use the same environment and sampling settings. Under the heterogeneous sampling scheme, governments act every 10 environment steps, whereas households act at every step. The capital-mobility curriculum is set to ϕk=min⁡(0.001k,1)\phi_{k}=\min(0.001k,1), and training data are collected in batches of five episodes. Based on training stability, we set both actor and critic learning rates to 3×10−63\times 10^{-6} for all algorithms to ensure a controlled comparison.

Decision periods. The heterogeneous sampling frequencies are motivated by practical experience: household saving and labor decisions are typically adjusted more frequently, whereas government fiscal policies are revised less often. We therefore allow households to act at every environment step and governments to act at a longer interval. We compared government decision intervals of 5, 10, and 20 steps and found that an interval of 10 achieved the best reward performance. Accordingly, we use 10 steps in the reported experiments.

Curriculum schedule. For N=2N=2 and N=5N=5, the training rewards become relatively stable at approximately 1,000–1,500 plotted training epochs. Guided by this behavior, we set$\phi_k=\min(0.001k,1),$ where kk is the training episode index. This schedule gradually increases the capital-mobility weight and reaches its full value after 1,000 episodes, allowing agents to learn first under weaker inter-group coupling before facing full competition. As the number of groups increases, the strength and duration of inter-group interactions change, so the curriculum rate may require further adjustment.

AuthorsOpenReview ↗

Looking forward to further discussions!

Dear Reviewer ZiXn,

Thank you for your encouraging evaluation and valuable feedback. We greatly appreciate your time and consideration.

We have provided further clarifications and additional experimental results to address your concerns. We would sincerely appreciate it if you could review our responses and share any remaining concerns. We hope our explanations will further strengthen your confidence in and support for our work.

Best regards,

The Authors

I thank the authors for their reply and additional work. My concerns have been partially addressed.

  • Thank you for adding the convergence plots and hyperparameter details. I am satisfied that the algorithm converges reliably, though its performance appears to suffer for increasing numbers of participants. The additional state-of-the-art comparison is certainly useful, though my comment was made primarily in the context of comparing how convergence and performance behaviour of different algorithms evolve when scaling the system.

  • The weaknesses I noted regarding the readability of the theoretical section do not appear to have been addressed.

My evaluation score remains the same. The experimental section and reproducibility have been strengthened, but the theoretical section remains excessively reliant on the appendix, somewhat impeding the reading of the main paper.

AuthorsOpenReview ↗

Response to the Reviewer's Follow-up Comments

Thank you for your follow-up feedback. We have further revised the theoretical presentation in the main text to improve readability and reduce reliance on the appendix. Please refer to the blue-highlighted revisions in Section 3.2 (p. 4). We hope these changes address your remaining concern about the theoretical section.

Reviewer_Jwy1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper extends single-government RL taxation (AI Economist, TaxAI) to multiple interacting economies. Within each group, the government and households play a leader–follower game. Across groups, governments compete for mobile capital through a Zodrow–Mieszkowski-style flow term. The proposed method, MGPPO, has three components:

  • two-timescale sampling;
  • an intra-group plus inter-group government critic;
  • a capital-mobility curriculum.

Compared with IPPO, MAPPO, and MAA2C, MGPPO lowers the Gini coefficients and raises GDP for N=2 (relative to IPPO). For N=5, it gives the lowest Gini but also the lowest GDP.

Critical strengths

Multi-government tax competition layered on top of leader–follower economies is a relevant and under-explored MARL testbed. Classical tax-competition theory provides a natural reference point. The environment could become a useful benchmark for mixed hierarchical and competitive MARL.

Critical weaknesses

W1. The empirical evidence for MGPPO's design is not yet sufficient.

  • Ablations. None of the three components is ablated, so it is unclear which of them drives the results.
  • Training seeds. The confidence intervals appear to come from evaluation seeds only.
  • Size of the gains. The gains are modest and mixed. For example, at N=2, MGPPO's GDP (5.37±0.12) overlaps with MAA2C's (5.35±0.12), and at N=5 MGPPO has the lowest GDP.
  • Baseline training. With a shared learning rate of 3e-6, several baselines produce nearly identical action means (Table 2). This raises the question of whether they were sufficiently trained.
  • Missing baselines. Comparing with bilevel or Stackelberg methods (e.g., TaxAI's method, Bi-AC, HAPPO) would make the evaluation more convincing.

W2. Game-theoretic analysis and method specification.

  • Equilibrium analysis. The problem is formulated as a game, but equilibrium quality (e.g., exploitability) is not evaluated. The capital tax increases with N, which is opposite to the race-to-the-bottom intuition behind the model; a discussion of this would be valuable.
  • Specification inconsistencies:
    • the curriculum schedule differs between Sec. 4.3 and Algorithm 1;
    • τˉ\bar\tau is defined in two ways, and only the capital-weighted version conserves capital;
    • it is unclear how the Q-style inter-group critic enters the GAE-based update.
Questions for authors
  1. Could you provide ablations of each component over multiple training seeds?
  2. Do the baselines share MGPPO's timescale and curriculum settings? Were their learning rates tuned?
  3. Could you report exploitability or best-response gains, and comment on why the capital tax increases with N?
  4. Which τˉ\bar\tau is implemented, and is capital conserved across groups?
3 comments
AuthorsOpenReview ↗

Authors' Rebuttal- Part 1

We thank the reviewer for the detailed comments. We have revised the empirical evaluation and clarified the method and experimental setup. Our point-by-point responses follow.


Concern 1: Could you provide ablations of each component over multiple training seeds? (W1: Ablations; Question 1)

A1: We added the following ablations for N=2N=2 and N=5N=5:

  • M1: the full model.
  • M2: a unified government critic that processes all available inputs.
  • M3: no hierarchical decision sampling.
  • M4: no curriculum learning.

Figure 3 shows that removing hierarchical decision sampling produces substantially lower government and household rewards in both settings. For N=2N=2, M1, M2, and M4 perform similarly, and removing the curriculum accelerates early learning. For N=5N=5, the full model reaches higher late-training government rewards than the unified-critic and no-curriculum variants, while their household rewards remain close. These results support the component choices in the evaluated setting and show that their benefits depend on group size and the reward being measured. For details, please refer to Section 5.2 (P9, lines 459), Figure 3 (P10).

To assess robustness to training initialization, we repeated the ablation experiments using two additional independent training seeds. The results show consistent overall trends across seeds and support the effectiveness of the proposed components. Please refer to Appendix D.3 (P 18, L 954).


Concern 2: Do the baselines share MGPPO's timescale and curriculum settings? Were their learning rates tuned? (Question 2)

A2: Yes. For the reported N=2N=2 and N=5N=5 comparison runs, all five algorithms use a government action interval of 10 steps, household actions at every step, and the same capital-mobility curriculum, ϕk=min⁡(0.001k,1)\phi_k=\min(0.001k,1). Collection is organized into batches of five episodes. The optimization rules remain algorithm-specific; in particular, BMFAC reuses transitions through replay and target critics. The learning-rate selection and BMFAC update settings are described in A4.

For details, please refer to Section 5.1 (P8, L381).


Concern 3: Could you report exploitability or best-response gains?( Question 3, W2: Equilibrium analysis.)

A3: We added a random-deviation-based local exploitability analysis. With all networks fixed, we evaluated three deviation settings using perturbation radii ϵ∈{0.10,0.01}\epsilon \in \lbrace 0.10, 0.01 \rbrace, 100 paired trials, and three evaluation seeds. At ϵ=0.01\epsilon=0.01, no sampled deviation improved the evaluated reward by more than 1%1\%. Larger perturbations generally reduced rewards or produced a government–household trade-off rather than a joint improvement. These results support the local stability of the learned outcome.

For details, please refer to Appendix~D.2 (P16, L863).


Concern 4: Why the capital tax increases with N? ( Question 3, W2: Equilibrium analysis.)

A4:

  • Trade-offs in capital taxation. A lower relative capital-tax rate may attract capital inflows and increase GDP, but it may also affect government revenue and, in turn, social welfare and the Gini coefficient. Governments therefore face trade-offs among attracting capital, maintaining fiscal revenue, and promoting equality. A lower capital-tax rate is not necessarily preferable.

  • Capital-tax actions as the number of groups increases. An interesting observation is that the learned policies exhibit higher capital-tax actions as NN increases. One possible explanation is that governments jointly adjust capital taxation and other fiscal policies, raising the average capital-tax level without necessarily widening relative tax differences across groups. In this case, higher tax actions would not necessarily lead to greater capital outflows. For details, please refer to Section 5.1 (P9, L452).


Concern 5: Which τˉ\bar\tau is implemented, and is capital conserved across groups?( Question 4, W2: Specification inconsistencies)

A5: Thank you for pointing this out. The unweighted definition, τˉt=1N∑n=1Nτt(n)\bar{\tau}_t=\frac{1}{N}\sum_{n=1}^{N}\tau_t^{(n)}, was mistakenly retained from an earlier draft. To ensure that cross-group capital flows conserve total capital, we use the capital-weighted average:

τˉk,t=∑n=1Nτk,t(n)Kt(n)∑n=1NKt(n)\bar{\tau}_{k,t}=\frac{\sum_{n=1}^{N}\tau_{k,t}^{(n)}K_{t}^{(n)}}{\sum_{n=1}^{N}K_{t}^{(n)}}

We have corrected this definition and standardized the notation for the capital tax rate as τk,t(n)\tau_{k,t}^{(n)} throughout the manuscript.

For the stated flow equation, this gives ∑nΔKt(n),flow=0\sum_n\Delta K_t^{(n),\mathrm{flow}}=0. Thus, cross-group reallocation conserves capital, while the separate accumulation process can change aggregate capital over time.

AuthorsOpenReview ↗

Authors' Rebuttal- Part 2


Concern 6: The gains are modest and mixed. (W1: Size of gains)

A6: Thank you for this comment. We agree that MGPPO's gains are not uniform across economic indicators. We assess learning performance primarily through the returns achieved under the specified reward functions. The higher rewards reported for MGPPO support the effectiveness of the proposed game-structured architecture in optimizing these objectives. Economic indicators provide complementary evidence about the resulting policies and should be considered jointly, since improving the reward does not necessarily improve every indicator.


Concern 7: With a shared learning rate of 3e-6, several baselines produce nearly identical action means (Table 2). This raises the question of whether they were sufficiently trained. (W1: Baseline training)

A7: We added the government and household reward trajectories for N=5N=5 to further demonstrate that the models were trained sufficiently. Together with the N=2N=2 results, these curves show that the rewards approach stable levels during the later stages of training.

For details, please refer to Figure 2 in Section 5.1 (P8 L415).


Concern 8: Comparing with bilevel or Stackelberg methods (e.g., TaxAI's method, Bi-AC, HAPPO) would make the evaluation more convincing. (W1: Missing baselines)

A8: Thank you for this question. To our knowledge, no established state-of-the-art baseline directly addresses our extended multi-group tax-policy environment. We therefore adapted BMFAC, a state-of-the-art method for single-group government–household interactions from TaxAI, as a comparison baseline alongside IPPO, MAPPO, and MAA2C. In this adaptation, each government learns to respond to inter-group competition using its individual information.For details, please refer to the blue-highlighted revisions in Section 5.1: economic outcomes (p. 8, line 391), reward analysis (P 8, line 424), and action analysis (P 8, line 436).

For N=2N=2, BMFAC achieves relatively low government and household rewards. For N=5N=5, it achieves the highest observed household reward but exhibits substantial fluctuations, while its government reward remains low. These results suggest that its effectiveness differs between agent types and group-count settings. Two factors may contribute to this behavior. First, BMFAC was developed for single-group leader–follower interactions, emphasizing household responses to government decisions. Its adaptation using individual government information may therefore be insufficient to capture cross-group strategic interactions. Second, BMFAC reuses historical samples through off-policy updates. As multiple agents’ policies evolve, these samples may become less representative of the current interaction dynamics, potentially complicating learning.

Concern 9: Why does the curriculum schedule differs between Sec. 4.3 and Algorithm 1? (W2: Specification inconsistencies)

A8: Thank you for pointing out the unclear presentation. The two expressions describe the same schedule under our experimental settings. The curriculum horizon is set to M=1000M=1000 and the maximum capital-mobility weight to ϕmax⁡=1\phi_{\max}=1. Therefore,

ϕk=ϕmax⁡min⁡(1,k/M)=min⁡(0.001k,1),\phi_k=\phi_{\max}\min(1,k/M)=\min(0.001k,1),

which matches Section 4.3. We have revised the training pseudocode to use the same explicit expression as the main text for clarity.

For details, please refer to Algorithm 1 in Appendix E (P21 L1087).

Concern 10: It is unclear how the Q-style inter-group critic enters the GAE-based update. (W2: Specification inconsistencies)

A10: Thank you for pointing this out. We have revised the pseudocode to clarify how the Q-style inter-group critic enters the GAE-based update. Please refer to the updated Algorithm 1 in Appendix E (P21 L1087).

AuthorsOpenReview ↗

Looking forward to further discussions!

Dear Reviewer Jwy1,

Thank you for your thoughtful comments. We sincerely appreciate the time and effort you have devoted to reviewing our work.

We have carefully considered each of your questions and provided further clarifications and additional experiments to address your concerns. If any points remain unclear or require further discussion, we would be more than happy to elaborate or provide additional analysis.

Your feedback has been invaluable in helping us improve our work, and we sincerely welcome any further comments or suggestions.

Best regards,

The Authors

Comments on the paper

2 comments
Area_Chair_UBz5OpenReview ↗

Reviewers Jwy1 and A6fe, please respond to the author by Nov 18 11:59pm AoE.

Dear Reviewers Jwy1 and ZiXn,

The author has replied to your reviews.

Does the author solve your concerns?

Please respond to the author and consider editing your rating.

The author-reviewer discussion phase will end November 18 at 11:59pm AoE.

Thanks.

AC

AuthorsOpenReview ↗

Authors' Global Response

Revision Summary

We sincerely thank all reviewers for their careful reading and constructive feedback, and the Area Chair for coordinating the discussion. Their comments have helped us improve both the experimental evaluation and the presentation of our work. The main revisions are summarized below.

1. Additional Experiments

  • Additional baseline: Adapted BMFAC from TaxAI to our multi-group environment and compared it with MGPPO, MAPPO, IPPO, and MAA2C in terms of economic outcomes, training rewards, and actions (Sec. 5.1; R1, R2, R3).
  • Ablation studies: Evaluated the joint government critic, hierarchical decision sampling, and curriculum learning for N=2N=2 and N=5N=5, using three independent training seeds in total (Sec. 5.2, Fig. 3, Appendix D.3; R1, R3).
  • Training stability and scalability: Added government and household reward trajectories for MGPPO at N=2,5,10,20,50N=2,5,10,20,50 to examine learning behavior as the number of groups increases (Sec. 5.1, Fig. 2, Appendix D.4; R1, R2). These results provide empirical evidence of training stabilization rather than a formal convergence guarantee.
  • Local deviation analysis: Added bounded random-action perturbation experiments across three evaluation seeds to examine local reward sensitivity. We clarify that this analysis is a local exploitability proxy, rather than an optimized best-response evaluation or an equilibrium guarantee (Appendix D.2; R1).

2. Methodological and Presentation Improvements

  • Contributions and novelty: Clarified the multi-group tax-competition environment and MGPPO's information structuring. (Sec. 4.2; R3).
  • Theoretical readability: Further revised Section 3.2 to improve the explanation in the main text and address the concern about excessive reliance on the appendix. (Sec. 3.2, p. 4; R2).
  • Comparison settings: Clarified baseline adaptations and the shared decision intervals, capital-mobility curriculum, and data-collection settings. (Sec. 5.1, Appendix C.2; R1, R2, R3).
  • Economic modeling: Corrected the average capital-tax definition. (R1).

3. Reviewer Feedback

We appreciate the reviewers' recognition of the problem's relevance and the additional evidence:

Overall


We are grateful for the reviewers' recognition of the significance, methodological design, and empirical contributions of our work:

  • Problem Significance: R1 (Jwy1) recognized multi-government tax competition as a "relevant and under-explored MARL testbed" and noted its potential as a benchmark for hierarchical and competitive MARL. R2 (ZiXn) acknowledged that our work "appears to address a real gap in the relevant literature."
  • Methodological Design: R2 commented that the "algorithmic and modelling choices are described clearly" and expressed "no substantive concerns about the paper's soundness." R3 (A6fe) recognized that the framework incorporates economic principles and that the proposed mechanisms are reasonable.
  • Empirical Evidence: R3, after reviewing our rebuttal and the additional BMFAC comparison, stated that "the empirical results seems to be strong" and acknowledged that combining the existing components in the proposed manner "certainly had some new merits as shown in the results."

We are grateful for the time the reviewers and Area Chair have devoted to our paper. We have conducted extensive additional experiments and analyses and substantially revised the manuscript to address the concerns raised by the reviewers. We believe that the added baseline comparisons, ablation studies, training stability analyses, and methodological clarifications have strengthened the empirical evidence and clarified our contributions. We hope these revisions address the reviewers’ concerns and would sincerely appreciate their reevaluation of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.