- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
AdaFleet is a system of LLM agents used to search quantitative signals for financial prediction. The goal is to find multiple signals that are useful for the prediction and are different from each other. AdaFleet essentially models the problem as a research problem and automates the research via an Orchestrator agent that allocates search and refinement tasks to sub-agents. Results show substantial increase in the number of signals found.
Critical strengths
- The problem is well-motivated.
- The Orchestrating system is thorough and is highly effective in maximizing the yield of signals.
Critical weaknesses
- Most importantly: the main metric in the paper experiments is discovery yield. It indeed looks important to have multiple useful signals that are not too correlated in pairs; but this metric is not sufficient to guarantee that the whole set of signals is strong. The primary objective should be more closely related to the ultimate prediction accuracy.
- An example to such a direct metric is Sharpe. I had to combine data from Table 2 and Appendix D3 in order to get explicit Sharpe values comparison between the methods, and found that AlphaSAGE beats AdaFleet on this metric.
- If I understand correctly, the metric only penalizes similarity (a) in pairs, not over the whole set together, and (b) only through correlation which misses non-linear mutual information.
- The intro is missing some context about the field, the problem and common methods.
Questions for authors
- Why is discovery yield the primary metric? Can you justify the method via more explicit downstream metrics as well?
- How would AdaFleet compare with incremental signal selection based on residual prediction?