- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces EPSBench, a benchmark for evaluating LLM agents on quarterly earnings forecasting. Agents estimate income-statement drivers using financial filings, earnings-call transcripts, and news, and a shared calculator converts these estimates into EPS forecasts. The authors evaluate six LLMs on 149 firm-quarters and compare their performance with analyst consensus and historical baselines. The results show that leading models perform similarly to analyst consensus, but all models have poorly calibrated prediction intervals.
Critical strengths
- The accounting-based framework is interesting. Breaking EPS predictions into individual financial drivers makes it easier to understand where forecasting errors come from.
- The paper provides useful experiments across six LLMs, including uncertainty calibration, information-source ablations, and harness comparisons.
Critical weaknesses
- The accounting-based framework is interesting. Breaking EPS predictions into individual financial drivers makes it easier to understand where forecasting errors come from.
- The paper provides useful experiments across six LLMs, including uncertainty calibration, information-source ablations, and harness comparisons.
Questions for authors
- Could you include stronger statistical or machine-learning forecasting baselines?
- How would the same LLMs perform if they directly predicted EPS without the accounting-based framework?
- How do you verify that the cited evidence actually supports the driver estimates?