Databox: Elevating SQL Debugging Agents with Instance-specific and Cost-Efficient Sandboxes
Abstract
The explosion of enterprise data and database-intensive applications has increased the complexity of SQL-based analytical and operational logic, raising the demand for automated query diagnosis and repair. Recent progress on SQL-based tasks has been driven by LLM-based agentic methods, which rely on execution-guided interaction loops with real database environments to iteratively inspect, reflect on, and refine candidate solutions. However, these agentic approaches typically require direct interaction with real databases, incurring high execution cost and latency on enterprise-scale applications and becoming impractical when privacy or deployment constraints restrict access to the underlying real data. To address these limitations, we propose Databox, an automatically constructed, instance-specific, data-specialized sandbox environment that serves as a drop-in replacement for real-database interaction. Databox is constructed with task-specific schema, diagnostic value, and consistency validation, enabling private, low-latency, and compute-efficient execution-guided SQL debugging without accessing the large original database. To further reduce reliance on large models and strengthen privacy guarantees, we introduce Databox-Local, which trains small language models for Databox construction and enables fully local deployment without external LLM calls. Experiments on BIRD-CRITIC-PG, spanning five widely used agent scaffolds and three backbone LLMs, show that Databox reduces optimizer-estimated execution cost by 65%-92%, with an average reduction of 85.2% relative to real-database interaction, while maintaining comparable task performance and slightly increasing the average success rate by 1.4%. To assess its practicality and generalizability, we further evaluate Databox on the repository-level benchmark BIRD-CRITIC-REPO, demonstrating the potential and adaptability to broader software engineering settings.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.