Visual Data Agent: Active Visualization and Perception of Data

LLM-based agents are increasingly used to analyze and audit structured data. However, existing agents perceive data predominantly through textual representations, such as sampled records, serialized schemas, or summary statistics. Serializing more records increases token cost and makes relevant information harder to locate in context, so agents typically inspect only a small portion of a large data source at a time. Furthermore, distributional data patterns that emerge across many records, such as a second peak in a series of values, are hard to capture. To bridge this gap, we present Visual Data Agent (VDA), an agent that actively plots data into chart-based views and captures patterns that sampled records and summary statistics can ignore. Starting with rule-based booting views, VDA actively explores data through calibrated charts paired with numerical summaries and task-specific agent notes, using coverage feedback to guide further inspection and visual memory to revisit earlier evidence. We evaluate VDA on seven public benchmarks, from ECG rhythms to satellite telemetry, and on DataPatternBench, a benchmark we construct that asks an agent to find distributional structure in real multi-table databases. On DataPatternBench, VDA improves detection F1 from 59.8% to 84.1% and discovery F1 from 43.1% to 78.0% over a strong coding agent (OpenHands) using the same base LLM, GPT-5.4. On six localization and classification benchmarks, VDA achieves an average absolute accuracy gain of 9.2% over OpenHands, averaging pooled accuracy gains across GPT-5 and GPT-5.4. In a controlled study with GPT-5.4, charts outperform numerical summaries in detection F1 (75.6% vs. 70.7%), highlighting the value of preserving distributional information beyond numerical summaries.