ChartSeer: A Benchmark for Inductive Reasoning over Charts

Chart understanding is an important capability of multimodal large language models (MLLMs) for analyzing and reasoning over data visualizations. Existing chart understanding benchmarks largely focus on deductive reasoning, where the question specifies what relation to evaluate, and typically involve charts with relatively few data points. As a result, they largely overlook the evaluation of inductive reasoning over charts, which requires models to discover latent regularities across sufficient data points without being explicitly given the specific regularity. To address this gap, we introduce ChartSeer, a benchmark for evaluating inductive reasoning over charts. We construct ChartSeer from 1,438 data-rich charts collected from diverse public datasets and chart repositories, each paired with its underlying tabular data. Based on the underlying data, we identify and verify latent regularities and construct 3,929 questions covering two complementary capabilities: rule induction for identifying latent regularities in charts and example inference for generalizing from these regularities to new cases. We evaluate 14 state-of-the-art MLLMs and compare them with humans, finding that current models remain substantially behind human performance. Further analysis reveals that visual perception and rule induction remain two major bottlenecks. We hope ChartSeer facilitates future research on the inductive reasoning capabilities of MLLMs for chart understanding.