- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents 65 source-linked mathematical errors produced by frontier models across 12 research-level problems in quantum information, quantum error correction, quantum complexity, and related fields. The authors also provide a comprehensive error analysis to investigate why these errors arise.
Critical strengths
- The collection of mathematical errors is a valuable resource, particularly as frontier models become increasingly capable and their remaining limitations harder to characterize.
Critical weaknesses
-
The paper’s fit for an AI conference is unclear in its current form. Although the collected examples are valuable, the analysis remains largely descriptive, offering limited insight into the underlying mechanisms that produce these errors or actionable strategies for preventing or reducing them.
-
The authors could strengthen the contribution by developing a benchmark for automatic mathematical error detection using the collected examples. Alternatively, these examples could serve as seeds for generating additional training data, enabling an investigation into whether targeted training reduces similar errors in LLMs.
Questions for authors
NA