- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
The authors analyze MAS of LLM agents acting within a financial institution and observe possible failures that can emerge from individual rational outcomes, such as trying to protect themselves. They develop a framework, called FRAIL, to analyze the fragility of isolated economic institutions populated by autonomous LLM agents. They show that such failures can happen and propose three mechanisms to alleviate such problems.
Critical strengths
The authors study a very interesting new cooperation problem of multi-agent LLMs within financial institutions. The setting is novel, and the experimental setting looks solid:
- The authors develop a simulation environment focused on risk in a financial environment, with dynamic financial trajectories that other related works have not explored in depth.
- The experimental analysis of the paper is strong; the authors show that failure exists, show that the chosen mechanism helps in alleviating the coordination problems and analyze reasons why such mechanisms help. They also ablate the environments with favorable or tighter conditions. They offer confidence bounds and different seed runs.
- The article is well written and clear. The images are clear and aid in understanding the setting. I also liked that the paper provided a glossary for CS people who might not be familiar with the finance background. These are small signals of attention to detail during the writing of the paper.
Critical weaknesses
I have a strong methodological issue regarding the parsing of the choice of actions in the experiments. The paper reports that it parses a “defensive intent” or a probability of choosing one or the other, and then samples using this probability. Yet, I argue that doesn’t correspond to correct parsing of the model's choice. In such discrete action models, the models in reality choose a single action. The fact that you then sample is something only your experimental setting does, but LLMs out there do clear actions instead. For example, if a model gives a 10% probability of withdrawal, having 12 agents means we expect at least one agent to withdraw even if all might give a 10% withdrawal possibility. If this were the case, then they would instead all choose not to withdraw. This would then feed into the next round, creating a bad feedback cycle.
I would like to see evidence with code of this not being relevant to raise my score. I don’t believe the “This design avoids brittle dependence on exact action parsing and, because the reports are graded rather than binary, provides an agent-level measure of defensive intent;” since modern models are not brittle anymore on action parsing if you prompt correctly.
Since your work is mostly experimental, evaluating the soundness of your code and setting is crucial for good judgment. I read the logs in your anonymous code, and indeed models choose 5 or 10 probability even when their reasoning is No redemptions occurred last round, and the institution appears comfortably solvent relative to promised obligations. I prefer keeping the deposit to earn the 1.114 maturity payment, with only a small withdrawal probability for unforeseen risk.
Another Issue, is that while authors offer ways to solve the coordination problem that they find within the financial systems, they offer no discussion on how their findings might offer insight into debugging or engineering some multi-agent system that can be alleviated by those issues. Especially in finance, where bad engineering can cause billions of dollars lost, we might care more about principled engineering of these systems instead of testing some mechanisms and empirically seeing that they make the problem less severe. The paper does the latter. This connects to my first question later when I ask the authors to explain how they came up with the mechanisms and the financial environments.
The model is still a highly abstract and formal model of how the specific institution there works; for example, the statement that “The institution fails immediately if it cannot pay a redemption request during processing,” is ok for the experiment, but might not be true of real financial institutions that the work is attempting to model. Real institutions may find other devices to finance themselves, especially if it's a state bank. But I agree that this might be out of scope for a single paper. Another drawback is that as a society we are already aware of the risks that automated trading/financial agents can have, e.g., Flash Crash. So, in my opinion, the risk that the paper is studying is quite limited. A more interesting setting could be blockchain or other newer financial institutions where regulation is less stringent.
I regard the last two issues as main reasons why I don’t lean towards a clear acceptance score even if the main methodological issue is resolved. I think the paper is well positioned, and it identified a true problem with LLMMAS with a couple of possible mechanisms to alleviate the problem, which makes me willing to vouch for a weak acceptance given resolution of the main methodological concern.
In addition, looking at the prompts, I see words such as [GAME PARAMETERS], which might tell the model that the environment is not “real”, which might produce some eval awareness problems. But since the model itself is abstract, as explained before, I don’t see this as a big issue in this evaluation scenario.
Single runs also all use the same model, even if models might differ between runs. I think this is a minor issue.
Another thing is that since you have seeded the runs, and the seed generates the original deposit (from the code), every model, mechanism and regime has the same amount deposited. This is also a minor issue, but might be helpful if you also ablate the amount deposited and see how much that influences the bank run.
Questions for authors
- I am curious to know why and how the authors selected the three experimental financial scenarios to test. There are also other cases where we can see such institutions failing. Finance is the modern version of some more classical cooperation dilemmas. In the same manner, I would be interested in how the authors chose the mechanisms.
- I am confused by “economic regime” appearing for the first time in section 4.3. What is it referring to?
- For example, I found
[INTERPRETATION REMINDERS] - Heavy redemptions in previous rounds may indicate elevated risk.in the prompt. Together with my methodological issue regarding probabilities, I believe the experiment may have mistaken results and bias the systems towards higher failures than expected. Can you share the logs of the Mechanisms too? With multiple rounds. - What share of exits came from reports below about 20%?
11 comments
Response to reviewer fdNB (Part2/4)
Research Contribution and Potential Engineering Implications
We humbly admit that our research has limited insight into debugging and engineering a multi-agent system. We focus on a different source of risk that is mostly parallel to the failure from engineering mistakes. The Coordination Fragility problem we studied can arise even when every agent operates correctly: individually protective withdrawals or redemptions change the decision environment faced by other agents and create system-level externalities. This is by the nature of these dilemmas and any kind of engineering advancement will not solve such a problem. Our contribution is to provide a reproducible setting in which LLM decisions alter a shared financial state, allowing us to trace collective failure and compare executable commitment arrangements.
We agree that the risks of automated financial activity are already recognized. Our unique contribution is to characterize coordination fragility more precisely. Our experiments deepen understanding of these dynamics and evaluate mechanisms that can mitigate them. Also, our abstract scenarios are not restricted to traditional institutions or particular regulatory regimes. We focus on shared dilemmas and coordination mechanisms rather than any specific institutional form, thus our abstract scenarios are inclusive of coordination challenges relevant to blockchain-based and other emerging financial arrangements.
Although detailed engineering guidance is not our objective, our findings may offer some useful design implications as well. For example, systems should provide authenticated communication channels and make commitments machine-verifiable from the start, helping agents coordinate more reliably. These are practical considerations for building more reliable multi-agent systems.
Responses to Your Questions
Q1: Why were these financial scenarios and mechanisms selected?
A1. Given the problem we try to tackle is the coordination fragility in multi-agent systems, we find financial decision scenarios most fit to such a topic and have the most practical implications. The principles we followed when we designed the experimental games are simple and clear: 1. The games must be representative of a family of real-world scenarios where coordination problem is an important issue and AI agents will possibly participate with autonomy in the foreseeable future. 2. The scenarios should be common, important and not difficult to understand. 3. The games should each capture distinct structures from real-world scenarios, depict different forms of coordination problems.
In a similar manner, we have chosen the 3 mechanisms that satisfy: 1. The mechanisms must resemble real-world mechanisms designed to mitigate the coordination problem. 2. The mechanisms must be distinct from each other in terms of how they affect action choices of the participants. 3. The mechanisms can be abstracted in a relatively simple fashion that enables the simulation results to be interpretable. The final chosen combination of scenarios and mechanisms follows all the principles above and together covers a broad scope of our research question.
The selected scenarios represent three distinct funding tasks: retaining existing funds in bank runs, renewing maturing funding in debt rollover, and assembling contributions to meet a goal in crowdfunding. Their different timing and funding needs allow us to study coordination across several financial structures. And the mechanisms cover compensated retention offers, centrally organized unpaid agreements, and participant-led coalitions, which each stands for a group of collective action protocols.
Q2: What does “economic regime” refer to?
A2. Thank you for pointing out this ambiguity. Here, an economic regime means a set of financial parameters. We create more favorable or tighter conditions by changing quantities such as asset returns, interest rates, available cash, and funding requirements. For example, the more favorable bank-run setting increases initial cash and asset returns. These are numerical changes to the environment. Section 4.3 describes the changes directly, Appendix D defines the term at its first appearance, and Table 6 in Appendix D.2 lists the parameter values.
Response to reviewer fdNB (Part3/4)
Q3: How do you address the prompt wording concern, and can you share multi-round mechanism logs?
A3. We agree that prompt wording is important to examine. We clarify how the reminder is used and then present the wording checks.
Risk reminder. The reminder is static and appears in withdrawal-decision prompts under M0–M3, including the first round when previous redemptions are zero. All four conditions also report the actual fraction redeemed in the previous round. The reminder describes a financial feedback present in the environment: redemptions can deplete cash and require discounted asset sales.
Rephrasing and economic descriptions. We ran 540 episodes using either a rewording that preserves the original instructions' meaning or an added optimistic or pessimistic economic description intended to nudge the agents' outlook. These tests change the text shown to the models while keeping the financial parameters and mechanism rules fixed. They also address reviewer qbAF's question about prompt variations.
Removing game and simulation wording. Another 180 episodes replaced fixed references to games and simulations with neutral financial descriptions. For example, [GAME PARAMETERS] becomes [FINANCIAL PARAMETERS], and “You are simulating the behavior of a single depositor” becomes “You are one depositor.” Financial rules and the risk reminder are retained.
After this wording change, M0 survival is 6.67% in bank runs and 0.00% in debt rollover, while commitment mechanisms continue to improve outcomes (Section 4.3; Appendices J.5–J.6). For your request for multi-round mechanism logs, the reading index (FRAIL_rebuttal_evidence/reading_index.md) lists 24 original-experiment episodes covering M0–M3 in all three environments for GLM and GPT-terra. Each includes the complete event stream for all participants, plus separately saved prompts and responses for participant 01. The two direct-action examples described above additionally include saved prompts and responses for every recorded call.
Q4: What share of exits came from reports below about 20%?
A4. We checked this directly in the recorded withdrawals. We matched each paid withdrawal to the same depositor’s reported probability in the same round across the 140 main-experiment bank-run episodes:
| Scope | Reported withdrawal probability | Share of paid withdrawals |
|---|---|---|
| All conditions | Strictly below 20% | 20.09% |
| All conditions | At or below 20% | 22.14% |
| M0 only | Strictly below 20% | 23.86% |
The denominators are 682 paid withdrawals across all conditions and 285 under M0. The paid_withdrawals_682.csv (FRAIL_rebuttal_evidence/checks/paid_withdrawals_682.csv) lists the reported probability and actual payment for the same depositor in the same round. It includes event locations and the indicators used for the percentages above. The check guide (FRAIL_rebuttal_evidence/checks/README.md) links to the supporting records and verification code. These counts describe the original experiments, in which models report probabilities and the simulator samples actions; the direct-action results above test explicit choices.
Response to reviewer fdNB (Part1/4)
Thank you for the careful inspection and constructive comments. In response to your major concern (methodological issue about the probabilistic action choice), we completed a full-scale robustness test to check results under direct action choices. Also, we acknowledge the limitation of this paper you mentioned in the official review, and would like to further articulate our research question and its potential implication. Finally, we will provide direct answers to your specific questions.
Direct-Action Results, Execution Code, and Traces
Response: We agree that modern LLMs can return explicit actions, and have removed the action-parsing justification in the manuscript. Inspired by mixed strategies in game theory, we ask participants to report action probabilities so that we can track how strongly they favor different choices as the financial situation changes. To ensure our results do not rely on the probabilistic action choice, we reran the experiments using direct action choice.
We changed the prompts so that models output an action directly, such as withdrawing or keeping a deposit, instead of outputting a probability for the environment to sample. The execution code then follows the selected action under the original financial and commitment rules. Reviewer qbAF raised the same concern, and we treated it as a priority in our additional evaluation. The results show that collective failures persist and commitment mechanisms improve aggregate outcomes. Below we provide the results, the execution code, and the recorded action sequences.
Aggregate results. The reported comparison in Table 14 gives the following mean normalized welfare on a percentage scale:
The comparison supports the main qualitative findings under direct action selection (Section 4.1; Appendix J.3). The table reports welfare, with 0% and 100% denoting the environment’s worst and best welfare reference points. Each cell summarizes 35 episodes, five per model; bold marks the largest mean in each row.
Execution code. The direct-action branch skips probability sampling. The following code in
bankrun/sim/runner.pyhandles the selected action under the original commitment constraints, then continues before the probability-sampling branch:Recorded trajectories. A recorded failure arises from valid discrete choices. In a GLM-5.3-Flash M0 bank-run episode, all 15 depositors initially choose
stay. Their subsequent choices develop as follows:These counts are depositors explicitly selecting
redeemin each round; actual payments remain subject to the financial rules. Every financial response in this baseline episode is a valid action output, with zero corrections and zero repairs. The bank’s failure therefore occurs in a trajectory of explicit action choices.In the accompanying direct-action M2 example, an agreement forms in round 1, commitments are announced, and the bank survives to maturity in round 12 without withdrawals. The direct-action walkthrough shows both episodes round by round and points to the relevant records. The additional evidence package ("FRAIL_rebuttal_evidence.zip") includes their saved prompts and responses for every recorded call, complete event streams, and metrics, together with the execution code.