ChipFactory: Capturing the World of Chip Design at Scale

Advancing AI for chip design requires models that can understand complex hardware, optimize designs, and assess their functional and implementation outcomes. Studying these capabilities calls for data that connect diverse design contexts and implementation choices with observable EDA backend feedback. However, previous work rarely combines design complexity, diversity, and backend feedback richness at scale. We present ChipFactory, a dataset of over 1 million design records covering complex hardware across diverse sources, abstraction levels, and implementation choices, with rich backend feedback from simulation through physical implementation. ChipFactory combines designs from generators and compilers, external datasets, human-designed accelerators, and software programs, relying on HLS together with AI agents to expand the latter two into design variants at scale. Reusable agent skills and an open harness let researchers build their own benchmarks from these design records. We instantiate 13 tasks spanning implementation prediction, behavioral prediction, ranking, and generation, and evaluate twelve LLMs, from state-of-the-art proprietary models to open-source and domain-specialized ones. Even the strongest model, Claude Fable 5.1, achieves an average score of only 58.4 out of 100, and the four smallest open-source models solve none of the generation problems. ChipFactory is open source to help the community expose the challenges models still face in reasoning across hardware representations and train specialized models for chip design.