- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes Vela, a JAX-based MARL framework for solving DSGE models. Built on the IPPO algorithm together with several engineering design choices, it is used to solve two representative DSGE models, the Kydland–Prescott RBC and the Smets–Wouters model. The authors estimate VARs on empirical data for ten economies, impose a one-time, unanticipated 5% shock on the model side, and read the response direction off the cumulative relative IRF 15 quarters after the shock, comparing the direction agreement of Vela's responses with those implied by the VARs. Vela achieves significantly higher direction agreement than the conventional numerical solution in eight of the ten economies, and the authors further report that Vela's responses are nonlinear under large simultaneous shocks. The paper's core contributions are the innovative use of MARL to solve DSGE models, the JAX-based engineering framework Vela, and higher IRF direction agreement together with more nonlinear IRFs on representative DSGE models.
Critical strengths
(1) The paper innovatively explores representing DSGE models with multi-agent systems (MAS) and using MARL algorithms to solve for global nonlinear solutions of DSGE models, and proposes a set of validation protocols.
(2) From an engineering perspective, it provides a complete solution framework; it adopts several engineering implementation details that are worth learning from and that make the algorithm work in practice, and it is implemented in high-performance computing languages such as JAX.
Critical weaknesses
(1) The authors claim that Vela is a nonlinear "solution" to DSGE models, but the validation is carried out through the direction-agreement rate of IRFs, and the authors take the conventional numerical method's lower score on this metric as evidence that Vela is better. This is a wrong conclusion obtained from a wrong validation procedure. Both DSGE models in the paper have numerical true solutions that can be computed within a minute; the reason the conventional solution does not score well on the authors' metric is not that it solves the model incorrectly, but that the model's structure is too simplified to explain real data well. This is a problem of the model's structure rather than of the method, and the authors conflate the two by using the model's explanatory power for real data to validate the performance of a solving algorithm. This means the results themselves are problematic: we cannot regard the output as a "solution" to the DSGE model, and hence the paper's central claim that "Vela is a nonlinear solution to DSGE models" does not stand.
(2) The paper lacks sufficient ablation experiments and comparisons against baselines: baseline algorithms are only mentioned in the main text but never compared experimentally, and robustness is not verified with respect to different economic parameters or different algorithm hyperparameters. As a result, we do not know which parameters Vela is sensitive to, or whether it is robust — questions that are central to its practical value.
(3) Other, smaller issues:
(3.1) In the code, several engineering optimizations of the RL setup (some mappings, clipping, and changes to the reward function) have, in my reading, already changed the structure of the SW model itself. Are the reported nonlinear responses then produced by these changes in model structure rather than by the solution? This makes Vela's results and the conventional numerical results not directly comparable.
(3.2) There is already a large body of research using MARL to solve DSGE models (e.g., TaxAI at AAMAS); the incremental value and novelty of this work are therefore low.
(3.3) Some of the solution settings are economically rather confusing (perhaps). For example, the projection method maps proposed actions back into the feasible set, whereas in my understanding one only needs to model the current effective cash-on-hand and let the household choose expenditure shares for the next period that sum to one — in which case the policy could never leave the feasible set, and this would also be closer to how people actually decide; the projection itself is counterintuitive. Furthermore, in the code the prices observed by the household are those set by the firms in the previous period, whereas under the standard market-clearing setup the clearing price should be formed within the current period, so the household should decide based on the current price set by the firms. Having firms set prices rather than letting prices be determined by market clearing is itself a very uncommon setup. Does market clearing still hold? Does Walras' law still hold? These questions are of first-order importance from an economic perspective.
Questions for authors
(1) Can the authors provide an accuracy report for Vela's solution — for example, Euler equation errors, the gap relative to the baselines, or other self-consistency metrics?
(2) Can the authors explain how this work differs from existing research such as TaxAI?
(3) Is Vela sensitive to economic parameters and to algorithm hyperparameters? Can the authors provide robustness checks or ablation results?
4 comments
Part 2/3
(W3.1) Limited Comparability with Conventional Smets Wouters
We modified the original Smets–Wouters specification to improve training convergence and reduce susceptibility to reward hacking in MARL setup. Nevertheless, the reviewer’s concern about limited comparability is well-founded. To address it, we constructed a revised conventional DSGE baseline incorporating the following features of Vela:
- No consumption habit formation.
- No variable capital utilization or fixed production costs, with separate capital and labor elasticities.
- Delayed dividend payments and profit taxation.
- A constant nominal money supply in place of the Taylor rule.
- Vela-style calibration and five shock processes.
We solved this revised baseline using first-order perturbation in gEconpy and repeated the empirical directional-accuracy evaluation described in Section 4.1. The results are reported below:
\hline \text{Country} & \text{Productivity} & \substack{\text{Labor}\\\\\text{preference}} & \text{Investment} & \substack{\text{Money}\\\\\text{preference}} & \text{Average} \\\\ \hline \text{Türkiye} & \substack{\mathbf{+33.33}\\\\{[+29.0,+36.9]}} & \substack{-19.64\\\\{[-24.3,-14.5]}} & \substack{\mathbf{+15.77}\\\\{[+10.7,+21.1]}} & \substack{\mathbf{+14.29}\\\\{[+9.0,+19.6]}} & \substack{\mathbf{+10.94}\\\\{[+8.3,+13.6]}} \\\\ \text{Italy} & \substack{\mathbf{+33.33}\\\\{[+29.0,+36.9]}} & \substack{\mathbf{+5.65}\\\\{[+1.0,+10.8]}} & \substack{\mathbf{+5.95}\\\\{[+0.8,+11.3]}} & \substack{\mathbf{+60.71}\\\\{[+55.7,+65.2]}} & \substack{\mathbf{+26.41}\\\\{[+23.8,+29.0]}} \\\\ \text{United States} & \substack{\mathbf{+41.07}\\\\{[+36.4,+45.1]}} & \substack{-29.46\\\\{[-34.3,-24.3]}} & \substack{\mathbf{+30.65}\\\\{[+25.3,+35.9]}} & \substack{\mathbf{+50.89}\\\\{[+45.6,+55.9]}} & \substack{\mathbf{+23.29}\\\\{[+20.6,+25.9]}} \\\\ \text{Japan} & \substack{\mathbf{+33.33}\\\\{[+29.0,+36.9]}} & \substack{\mathbf{+14.29}\\\\{[+9.0,+19.6]}} & \substack{-14.58\\\\{[-18.8,-9.8]}} & \substack{\mathbf{+2.68}\\\\{[-1.8,+7.7]}} & \substack{\mathbf{+8.93}\\\\{[+6.3,+11.6]}} \\\\ \text{South Korea} & \substack{\mathbf{+37.20}\\\\{[+32.3,+41.6]}} & \substack{-8.04\\\\{[-12.7,-3.0]}} & \substack{\mathbf{+32.14}\\\\{[+27.4,+37.3]}} & \substack{\mathbf{+44.35}\\\\{[+39.1,+49.7]}} & \substack{\mathbf{+26.41}\\\\{[+23.8,+29.1]}} \\\\ \text{Australia} & \substack{\mathbf{+41.07}\\\\{[+36.4,+45.1]}} & \substack{\mathbf{+16.07}\\\\{[+10.7,+21.3]}} & \substack{\mathbf{+3.87}\\\\{[-0.7,+8.9]}} & \substack{\mathbf{+22.62}\\\\{[+17.7,+27.9]}} & \substack{\mathbf{+20.91}\\\\{[+18.2,+23.6]}} \\\\ \text{Canada} & \substack{\mathbf{+41.07}\\\\{[+36.4,+45.1]}} & \substack{\mathbf{+9.82}\\\\{[+4.9,+15.1]}} & \substack{\mathbf{+14.88}\\\\{[+9.8,+20.2]}} & \substack{\mathbf{+34.52}\\\\{[+29.3,+39.9]}} & \substack{\mathbf{+25.07}\\\\{[+22.4,+27.7]}} \\\\ \text{France} & \substack{\mathbf{+33.33}\\\\{[+29.0,+36.9]}} & \substack{\mathbf{+36.61}\\\\{[+31.3,+41.9]}} & \substack{\mathbf{+18.45}\\\\{[+13.3,+23.8]}} & \substack{\mathbf{+60.71}\\\\{[+55.7,+65.2]}} & \substack{\mathbf{+37.28}\\\\{[+34.7,+39.8]}} \\\\ \text{Germany} & \substack{\mathbf{+33.33}\\\\{[+29.0,+36.9]}} & \substack{-44.94\\\\{[-49.6,-39.8]}} & \substack{\mathbf{+4.46}\\\\{[-0.2,+9.6]}} & \substack{\mathbf{+23.21}\\\\{[+17.9,+28.6]}} & \substack{\mathbf{+4.02}\\\\{[+1.4,+6.7]}} \\\\ \text{United Kingdom} & \substack{\mathbf{+33.33}\\\\{[+29.0,+36.9]}} & \substack{-81.85\\\\{[-85.6,-77.4]}} & \substack{\mathbf{+30.95}\\\\{[+25.8,+36.3]}} & \substack{\mathbf{+9.82}\\\\{[+4.5,+15.2]}} & \substack{-1.93\\\\{[-4.6,+0.7]}} \\\\ \hline \text{Average} & \substack{\mathbf{+36.04}\\\\{[+33.4,+38.7]}} & \substack{-10.15\\\\{[-35.1,+14.8]}} & \substack{\mathbf{+14.26}\\\\{[+3.6,+24.9]}} & \substack{\mathbf{+32.38}\\\\{[+17.3,+47.5]}} & \substack{\mathbf{+18.13}\\\\{[+9.5,+26.8]}} \\\\ \hline \end{array}$$ The table shows that Vela’s average empirical directional-accuracy advantage increases from 9.69 percentage points against the original Smets–Wouters baseline to 18.13 percentage points against the revised baseline. Thus, under this metric, the original comparison was more favorable to the conventional baseline. **(W3.2/Q2) How does Vela differ from TaxAI?** Compared with TaxAI, which adopts an agent-based modeling (ABM) approach grounded in the Bewley–Aiyagari model with up to $10^5$ households agents, Vela focuses on DSGE environments with representative agents and multiple coupled economic decisions and constraints. We aim to develop a reusable framework across economic models, while TaxAI focuses exclusively on optimal taxation. We view these contributions as complementary and will expand our related-work section to cover a broader range of MARL applications to economics.Part 3/3
(W3.3) Does Walras law hold?
Budget accounts (namely for the household) are balanced by design via @constraint primitive:
Here, denotes labor income, capital rental income, lagged nominal profits payed as dividends, and the current price level. and denote real consumption and investment expenditures, while is unspent income retained as money. Figure 7 shows household accounting discrepancies of order across the displayed shock trajectory over 256 parallel environments, establishing accounting consistency.
Next, we compare real production value with household income, the counterpart of income-based GDP in the Smets–Wouters model:
Although this relationship is not imposed by construction, it holds approximately in our simulations, with a one-period overproduction spike followed by rapid adjustment (Figure 8). Together, these results indicate agreement between production-, income-, and expenditure-based GDP for the learned policies.
Futhermore as reward function implies penalizing agents for proposing actions that violate market clearing contraints we investigated how this penalty term progresses over the course of PPO training. Figure 9 shows a sustained decline, indicating that proposed actions require progressively smaller feasibility corrections.
Thank you for the additional experiments and explanations; I appreciate the effort. The new results address most of my concerns, and I agree they show that Vela is robust within its current framework.
I still lean toward keeping my current score, mainly because I am not yet convinced that the accuracy evidence establishes Vela recovers the correct global solution. The perturbation-based residual is informative about local optimality, but Euler-equation errors and a comparison of policy functions against existing numerical solutions would speak more directly to this question. SW is a heavily studied model with a rich literature on higher-order perturbation and global solutions, so such a comparison should be relatively inexpensive and would considerably strengthen the paper.
On TaxAI: my mention was about model complexity. SW has no infinite-dimensional state, whereas TaxAI solves an Aiyagari model with taxation and the MARL-BC paper at last year's AAMAS solves Krusell–Smith—both substantially more complex objects for a solution algorithm. I agree that studying IRFs from MARL solutions is novel; that is precisely why I would like to see stronger evidence for their accuracy.
Part 1/3
We thanks reviewer for their time and constructive feedback.
Addressing weaknesses and questions
(W1/Q1) Solution Self-Consistensy Metrics and First-order optimality
Thank you for highlighting the need for a deeper analysis of solution self-consistensy metrics. Do do so, we performed additional experiments on the first-order optimality. The first-order condition (FOC) requires that, at an interior optimum, a small feasible change in an action cannot improve expected lifetime utility: ∂aj∂Qπ(s,a)=0. We assess local optimality by perturbing each action coordinate by h=±0.01 and simulating H=80 periods, with all agents following their frozen policies thereafter. We average return improvements over two stochastic continuation branches, using matched random draws for the original and perturbed trajectories. For each parallel environment, we compute Rj=max!(0,h+ΔJ+,h−ΔJ−). Figure 1 tracks these residuals during training for five household and three representative-firm action coordinates across six random seeds. Residuals are averaged over evaluated states within each run; the curves show the across-run median, with interquartile bands. Residuals decline by aproximately four magnitudes from 101 to 10−3, indicating substantial improvements in local policy optimality through multi-agent self-play. Stochastic estimation and numerical constraint enforcement can contribute to the remaining residuals.
(W2/Q3) Algorithm Hyperparameters Ablation Study
We compare training configurations using agents’ base economic rewards, excluding constraint-violation and action-magnitude penalties from the evaluation metric.
We examine the constraint-violation penalty, which discourages infeasible action proposals (Figure 2), and the action-magnitude penalty, which discourages abrupt adjustments (Figure 3). Both ablations show that removing the respective penalty substantially reduces the economic rewards attained by households and firms.
Appendix E additionally examines the historical-policy population buffer size (Figure 4) and the number of parallel training environments (Figure 5). Within the tested configurations, larger policy populations and more parallel environments improve training performance.
(W2/Q3) Economic Hyperparameters Ablation Study
To assess sensitivity to economic calibration, we repeated the empirical directional-accuracy evaluation across parameter values within the training domain-randomization ranges. We varied one parameter at a time while holding the others at their range midpoints. Figure 6 reports the country-averaged accuracy difference: Δc=100(AccuracyVela,c−AccuracySW2002,c) Vela’s empirical directional-accuracy advantage remains positive across all tested calibration values, although its magnitude varies by parameter. The advantage is relatively insensitive to the shock persistence parameters ρ, but varies substantially with the production elasticities αK and αL, with the largest gains occurring at intermediate values. Lower capital depreciation rates δ yield smaller gains, while the advantage peaks at intermediate depreciation rates. Overall, these results suggest that Vela’s empirical advantage persists across the tested calibration ranges, with greater sensitivity to production and depreciation parameters than to shock persistence.