Evidence-Guided Harness Evolution for Articulated CAD Agents

Articulated computer-aided design (CAD) agents rely on modeling knowledge written into their harnesses by hand, and harness evolution could acquire this knowledge from experience without updating the base model. Examining what conventional harness evolution learns, we find that most of its early gain comes from environment knowledge shared across tasks; adding this knowledge to the initial harness by hand recovers the gain. The modeling experience accumulated afterwards helps less reliably, for two reasons: it can reach tasks it does not fit (an applicability gap), or the agent can follow it and still carry out the operation incorrectly (a knowing–doing gap). The two failures look the same in a task score but call for different revisions. We therefore propose evidence-guided harness evolution, which localizes failures before revising experience. The outer loop tests hypotheses about which experience to add or change through geometric and kinematic checks against training references and targeted trials. Evidence traces each use of an experience through four questions—whether it fit the task, was followed, was carried out correctly, and improved the outcome—and the first that fails decides what to revise. Applicability conditions are narrowed or widened, and operations the agent repeatedly gets wrong become programs checked against local contracts. Starting from an environment-informed harness, so that gains measure what evolution adds beyond it, our method raises the test score from 0.626 to 0.743, above conventional evolution (0.683) and prior harness optimizers. The frozen harness also improves unseen categories, another dataset, and other base models. These results support our view that the scope and realization of experience should be revised with evidence from its use.