Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs

Large language models (LLMs) are becoming increasingly capable mathematical collaborators, but static benchmarks are no longer sufficient for evaluating progress: they are often narrow in scope, quickly saturated, and rarely updated. This makes it hard to compare models reliably and track progress over time. Instead, we need evaluation platforms: continuously maintained systems that run, aggregate, and analyze evaluations across many benchmarks to give a comprehensive picture of model performance within a broad domain. In this work, we build on the original MathArena benchmark by substantially broadening its scope from final-answer olympiad problems to a continuously maintained evaluation platform for research mathematics with LLMs. We regularly evaluate new models, review their execution, and publish detailed analyses that inform updates to benchmarks and evaluation protocols. MathArena now includes three research benchmarks: ArXivMath for final-answer questions, BrokenArXiv for responses to false mathematical claims, and ArXivLean for formal proof generation in Lean. Currently, GPT-6 Astra is the best-performing model on MathArena, achieving 88% expected performance on the non-deprecated benchmarks.