PDAgent-Bench: Characterizing, Grounding, and Architecting LLM/VLM Agents for VLSI Physical Design

Large Language Models (LLMs) and vision-language models (VLMs) have shown remarkable success in the front-end design of Very Large-Scale Integrated Circuits (VLSI), yet their capabilities for VLSI physical design (PD) remain significantly underexplored. One primary cause is the lack of standardized benchmarks for evaluating agentic PD workflows that require high-dimensional, multi-stage optimization under strict design constraints, coordinated interaction with diverse Electronic Design Automation (EDA) tools, and iterative refinement. This work introduces PDAgent-Bench, a comprehensive and multi-dimensional benchmark for evaluating LLM/VLM-based agents across the physical design stack. PDAgent-Bench integrates both task-level assessment and workflow-level execution. The benchmark suite contains 353 curated problems that combine conceptual questions with real-world industrial artifacts, with expert-validated references and executable solutions. These tasks cover five key capability dimensions: foundational knowledge, report comprehension, root-cause analysis, script generation, and full-flow implementation. In addition, the benchmark provides a unified, human-aligned agentic PD workflow framework that enables closed-loop evaluation of holistic PD in realistic EDA environments. Experiments on state-of-the-art models reveal that while modern LLMs/VLMs perform competitively on conceptual tasks (e.g., Opus5 achieves 74.8% on root-cause analysis), they remain substantially limited in tool-centric execution and long-horizon (Opus5 achieves 39.8% on Script-ICC2), multi-stage reasoning. Our studies further show that human-skill-enhanced agentic workflows significantly improve end-to-end PD performance. PDAgent-Bench establishes a standardized, reproducible, and realistic evaluation framework for advancing LLM/VLM-driven holistic physical design automation.