- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes PI-MAT, a parasite–immune multi-agent trading framework aimed at reducing the “pseudo-multi-agent” problem, where agents with different roles may still rely on similar information and generate highly correlated strategies. PI-MAT combines trend-following, mean-reversion, and event-driven agents with distinct information perspectives, and introduces strategy evolution and an online performance-based arbiter to dynamically suppress weak signals and reallocate capital. The framework is evaluated on daily AAPL data from 2020–2023 with several ablations and baselines. The authors report an annualized net return of 18.72%, a Sharpe ratio of 0.729, and a maximum drawdown of 28.61%, with a 3.52 percentage-point improvement in annualized return over the price-only baseline.
Critical strengths
-
The problem formulation is interesting:agents may have different roles but still rely on similar information and produce highly correlated strategies.PI-MAT addresses this by separating information perspectives across trend-following, mean-reversion, and event-driven agents and by introducing adaptive strategy evolution.
-
The ablation is useful:the paper compares PI-MAT with NoEvo, NoImmune, SharedInput, and EqualWeight variants, and also reports split-period and rolling-window results. These experiments help isolate the contributions of the strategy-evolution and adaptive-allocation components.
Critical weaknesses
-
The empirical evaluation is too limited to support the paper’s broader claims. The main out-of-sample evaluation is conducted only on daily AAPL data from 2020–2023. For a framework that claims to improve multi-agent trading under non-stationary markets, evaluation across more assets, market regimes, and longer periods is important to establish generality.
-
Some reported results raise concerns about experimental validity and interpretation. The event-driven proxy achieves an unusually high standalone annualized return of 224.84%, a Sharpe ratio of 8.05, and a maximum drawdown of only 0.03%, but the paper does not provide enough detail to rule out data leakage or overly favorable assumptions. It is also difficult to reconcile such a strong expert with the much weaker final PI-MAT performance of 18.72% annualized return and 0.729 Sharpe, which raises questions about the effectiveness and interpretation of the arbiter.
Questions for authors
-
Could the authors provide the exact construction of the event-driven proxy, including its data sources, timestamps, and the information available at each trading decision? Its standalone performance (224.84% annualized return, Sharpe 8.05, and 0.03% maximum drawdown) is unusually strong, so a clearer explanation of how look-ahead bias or data leakage is prevented would materially affect my assessment.
-
How should the much weaker final PI-MAT performance (18.72% annualized return, Sharpe 0.729) be reconciled with the very strong event-driven expert? Please clarify whether portfolio constraints, risk controls, or the arbiter’s allocation mechanism are responsible for this gap, and whether an oracle or event-only comparison under the same portfolio constraints is available.
-
The paper describes the 2020–2023 evaluation as preregistered. Could the authors provide the preregistration record and clarify which hypotheses, hyperparameters, baselines, and evaluation procedures were fixed before examining the out-of-sample results?
3 comments
Response to Questions For Authors
Questions For Authors1 & 2:"Could the authors provide the exact construction of the event-driven proxy, including its data sources, timestamps, and the information available at each trading decision? Its standalone performance (224.84% annualized return, Sharpe 8.05, and 0.03% maximum drawdown) is unusually strong, so a clearer explanation of how look-ahead bias or data leakage is prevented would materially affect my assessment. How should the much weaker final PI-MAT performance (18.72% annualized return, Sharpe 0.729) be reconciled with the very strong event-driven expert? Please clarify whether portfolio constraints, risk controls, or the arbiter’s allocation mechanism are responsible for this gap, and whether an oracle or event-only comparison under the same portfolio constraints is available."
Response to Questions For Authors 1 & 2:"We thank the reviewer for identifying this interpretive issue. Our audit confirmed that the event proxy’s reported 224.84% annualized return, 8.05 Sharpe ratio, and 0.03% maximum drawdown are affected by look-ahead bias. Its evaluation period also differs from PI-MAT’s 2020--2023 evaluation; the results are therefore not directly comparable with PI-MAT’s reported 18.72% annualized return and 0.729 Sharpe ratio. We will recompute the results over a common evaluation period using signals based only on information available at each decision time, and update table 1."
Questions For Authors3:"The paper describes the 2020–2023 evaluation as preregistered. Could the authors provide the preregistration record and clarify which hypotheses, hyperparameters, baselines, and evaluation procedures were fixed before examining the out-of-sample results?"
Response to Questions For Authors 3:"The preregistration record for the paper’s 2020--2023 evaluation has been submitted as supplementary material."
I thank the authors for their responses and additional experiments. After considering the revisions, I will maintain my original rating of 2 (Weak Rejection).
Response to Critical Weaknesses
Critical Weaknesses1:"The empirical evaluation is too limited to support the paper’s broader claims. The main out-of-sample evaluation is conducted only on daily AAPL data from 2020–2023. For a framework that claims to improve multi-agent trading under non-stationary markets, evaluation across more assets, market regimes, and longer periods is important to establish generality."
Response to Critical Weaknesses1:"We thank the reviewer for this comment. To assess robustness across assets and market conditions, we have added long-horizon evaluations of AAPL, NVDA, and MSTR using available data from 2005 to 2023, divided into five periods: pre-crisis, the global financial crisis, post-crisis recovery, 2016--2019, and 2020--2023. All experiments use a consistent transaction-cost convention and evaluation metrics, and report PI-MAT’s annualized return, Sharpe ratio, and maximum drawdown. The results show that PI-MAT remains profitable under several heterogeneous conditions: AAPL records positive returns in every evaluated period, including the 2008--2009 crisis and the 2020--2023 test period; MSTR performs strongly before and during the crisis; and NVDA remains profitable during the recovery and subsequent growth periods. These results suggest that the proposed heterogeneous-agent coordination and performance-based capital-allocation mechanisms remain effective across changing market conditions."
Critical Weaknesses2:"Some reported results raise concerns about experimental validity and interpretation. The event-driven proxy achieves an unusually high standalone annualized return of 224.84%, a Sharpe ratio of 8.05, and a maximum drawdown of only 0.03%, but the paper does not provide enough detail to rule out data leakage or overly favorable assumptions. It is also difficult to reconcile such a strong expert with the much weaker final PI-MAT performance of 18.72% annualized return and 0.729 Sharpe, which raises questions about the effectiveness and interpretation of the arbiter."
Response to Critical Weaknesses2:"The event-proxy signal was aligned with its contemporaneous return, introducing look-ahead bias and producing the reported 224.84% annualized return. To ensure a fair comparison, we will recompute the results over a common evaluation period using signals based only on information available at each decision time, and update table 1."