- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces a new source-grounded benchmark and experimental framework for evaluating the relability of finanical LLMs at the human-AI system level. The key aspect of the decision making process that this framework seeks to address is how a LLM response influences the recipient of the response; it is possible for an incorrect LLM response to prompt the user to switch to an incorrect response with an error differing from that inherent to the LLM response. The key contributions of this work include an evaluation of how model error is propagatd or mitigated at the decision-making level, meausrement of AI reliance and a source-grounded finanical failure benchmark.
Two 7B models, Qwen2.5-7B and Mistral-7B, were evaluated on the proposed Stress-Test v2 benchmark. Both models were shown to be unable identify the target category for any of the 20 controlled failure types included.
Critical strengths
The question being addressed is one that should be of interest to many AI practitioners, particularly as AI is increasingly used and may influence decision-making. In highly regulated industries, such as finance, this aspect carries even greater importance as even subtle shifts or failure modes introduced due to the addition of AI to the decision-making workflow could carry significant consequences, both on the basis of outcomes and regulations.
To that end, the construction of FinAgent-Reliance appears to be sound and comprehensive in how it seeks to evaluate and benchmark the LLM-human system. The stratification of takss by task types, failure variants and assistance conditions provides a fairly holsitic evaluation of the AI-human system.
Critical weaknesses
I believe there are two glaring weaknesses in this manuscript, both somewhat linked to the readability.
-
The main maunscript references too many numbers and terms with little to no grounding for a reader. A good example of this is the first dicsussion of the conditions on Line 168; these are central to the construction of the proposed benchmark but the only visible reference to them. before this point is in Figure 1. These should be clearly stated and justified due to their importance to the subsequent description and discussion. Along a similar line, the reader is not given any examplars from Gold Core v1 or the Financial-Agent Failure Library. This lends a sense of arbitrariness to these objects despite the justification presented for their uses in the paper. More examples of this are provided in the next section (in the form of questions).
-
The manuscript is structured confusingly in parts. The abstract contains several metrics which read like a recap of a technical report rather than substantiating the key takeaways. Aspects neeeded to understand Sections 2 and 3 are explained in Section 5. While some of writing is a personal stylistic choice, an effort has to be made to enable a reader to develop and follow the reasoning presented by the authors as they read the paper as it is presented.
Questions for authors
- The term "parent-family" or "parent" is used multiple times in the manuscript. What does this mean?
- The discussion in Section 7 is very general. In its current form, it is difficult to grasp what the numbers mean and how they link to the proposed objectivs of this work. 3, What would be needed for the authors to make more general remarks about LLM-human systems?
- Results and metrics are mostly scattered trhough the text; this makes it difficult to reference these results and subsequently, follow the discussion.
- What is the "Experimental Stimuli v1"?
We thank the reviewer for the detailed suggestions on the presentation of the framework, benchmark composition, results and examples. They identified places where the original manuscript was insufficiently clear.
Experimental conditions C1–C4 We agree that the conditions must be defined before they are used. Section 3 now introduces them: C1 allows human-only reconsideration; C2 shows a fixed AI response without citations; C3 adds optional inspection of cited evidence; C4 adds a mandatory checklist and a neutral challenge prompt and, in Pilot v2, requires evidence inspection before submission. Initial judgements are locked before the intervention, the AI response is identical across C2–C4, and C3 and C4 share the same frozen citations.
Parent families and benchmark construction Section 4 now defines a parent task as a validated financial question with a correct reference response and authoritative evidence. A controlled variant introduces exactly one material error while preserving the question and evidence. Stress-Test v2 contains 12 parent families: 12 correct responses and 35 controlled failures, 47 records in total. These are not 47 independent tasks, so uncertainty is estimated with parent-family cluster bootstrapping. The separate roles of Gold Core v1, the Human-Validated Expansion Base, Experimental Stimuli v1 and Stress-Test v2 are now explicit in Section 4 and Appendix B.
Experimental Stimuli v1 The two resources serve different purposes. Experimental Stimuli v1 (24 tasks) supports the human pilots, measuring changes between initial and final judgements under C1–C4. Stress-Test v2 is a failure-enriched diagnostic benchmark for automated error detection and classification. Using different resources prevents a direct item-level comparison between automated error recognition and human error inheritance. Sections 5.1 and 7 now discuss this limitation explicitly.
Presentation and interpretation of results The Results and Discussion now separate binary error detection from exact failure-category classification. Qwen2.5-7B achieves balanced accuracy of 0.576, and Mistral-7B 0.475 when unparseable outputs are counted as errors. Both have zero exact-category recall on 20 controlled failures in seven categories. This does not mean all 20 errors went undetected; some were flagged as errors but assigned the wrong category. [If 32B is reported: Qwen2.5-32B-Instruct reaches 0.775 and correctly classifies 9 of these 20; because it ran in BF16 and the 7B models were 4-bit quantised, we do not attribute this to scale.]
The human studies are described as exploratory protocol-validation pilots, not confirmatory tests of intervention effectiveness. In Pilot v2, evidence was viewed in 38/64 C3 trials and 64/64 C4 trials, where inspection was required, while final accuracy was 76.6% in both conditions. These observations are descriptive and do not show that either interface changed decision quality.
Representative benchmark examples We agree that real records demonstrate source grounding better than schematic examples. Appendix B now replaces the synthetic illustrations with verified records from the frozen Gold Core and Failure Library. Each gives the question, the SEC filing accession numbers, the extracted figures, the validated Gold answer and, for the failure record, the controlled modification and its category. Appendix A.4 additionally presents two observed participant cases on real filings (Goldman Sachs/BNY Mellon and American Express/Discover).
Generalisability Sections 4, 7 and 8 now discuss generalisability more fully. Stress-Test v2 has only 12 independent parent families, several categories have 2–4 examples, and the pilots are small with repeated observations per participant. We therefore present the findings as diagnostic evidence about the tested configurations and the feasibility of the measurement protocol, not as general conclusions about financial LLMs or human–AI decision-making.
We appreciate the reviewer’s suggestions, which have helped us present the framework and its limitations more clearly.