ChipReasoning: Benchmarking Evidence Discipline in Agentic Chip Implementation

Engineering decisions in chip implementation depend on tool reports. Report selection is therefore part of the task: an agent must inspect the right evidence and keep its claims within the scope of that evidence. We introduce ChipReasoning, a benchmark comprising 170 cases organized into 85 counterfactual pairs across six chip-engineering domains. Each case provides six labeled but initially hidden report packets under a fixed inspection budget. The two cases in each pair share the same request and visible labels, but differ in one decision-critical packet. We evaluate both the inspection record and the submitted engineering disposition using a seven-component Process score. Significantly, across 11 configurations, Process scores range from 42.22 to 71.60, with 10 of 11 configurations outperforming the strongest deterministic, content-blind inspection baseline, which scores 50.41. While the leading configurations successfully inspect the changed packet in more than 97% of pairs, the highest both-correct pair rate remains only 45.9%. These results strongly emphasize that the remaining errors are concentrated in translating retrieved evidence into the final engineering commitment.