- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes RAASEC, a filter that decides which individual actions from an LLM trading agent to execute. Each candidate action, sampled under 10 perturbation prompts, is scored by five confidence and risk measures, combined by a geometric mean, and the top fraction is executed. On a 2025 backtest of CSI 300 stocks with Qwen3-32B, RAASEC reports 52.57% annualised return and a Sharpe ratio of 2.11, outperforming TradingAgents and FinAgent.
Critical strengths
Action-level selective execution for LLM trading agents is a practically relevant problem, and the appendix documents the data pipeline and prompts in detail.
Critical weaknesses
-
Only one 11-month backtest, one market and one base model (Qwen3-32B) are used, the reported number is the "best run" with no variance or significance tests, and no transaction costs (commission, stamp duty, slippage) are modelled.
-
The paper criticises TradingAgents and QuantAgent because their risk judgment "still relies on LLM reasoning", but RAASEC's MRBC is itself an LLM-as-judge, and the final trading agent can still adjust, defer or reject actions.
-
Qwen3 was released in April 2025, so its pretraining data may overlap the early 2025 test period, but the paper did not discuss about this, while still claiming to "thoroughly eliminate look-ahead bias".
-
The results are too good to be true. A 52.57% annualised return and 2.11 Sharpe from a 32B model in a market the authors call especially hard is extraordinary, and FinAgent's active-day count implies 234 trading days (163 / 69.66%) while every other row implies 232.
-
Writing errors and contradictions:
- Threshold direction is reversed between Sec. 4.3 and Sec. 5.2.
- The same ablations report different numbers in Tables 3 and 4 (e.g., "w/o RFBC": 44.81% vs. 32.24%).
- Figure 1's caption refers to a nonexistent "Confidence Filtering Agent".
- Typos include "sort-of-states methods", "Adapative", "Assesing" and "MRPC".
Questions for authors
Please see the weakness section