Expert-Grounded Synthetic Data for Cognitive Biases: Controlled Generation and Blind Evaluation of Construct Fidelity

Reliable systems for identifying cognitive-bias mechanisms require datasets whose labels are psychologically faithful, domain-grounded, and able to distinguish neighboring constructs. Large language models can scale candidate generation, but generic prompting may produce fluent items that express a different mechanism from the intended target, creating mechanism-level label noise. We study this problem using a private, psychologist-curated Cognitive Bias Domain Atlas containing 229 cognitive biases, 30 application domains, and 2,385 curated bias–domain relationships whose records may include expert definitions, domain-specific justifications, and reference examples. We compare three conditioning packages over 31 matched relationships spanning 10 biases and five domains: Public, using a literature-grounded definition; Atlas-DJ, using an expert definition and domain-specific justification; and Atlas-Full, which additionally supplies an expert reference example with no-copy and overlap safeguards. Across 93 generation cells, the system produced 930 candidate survey items, of which 857 were eligible for condition and target-blind review by a psychology expert. Primary inference is conducted at the relationship level. Relative to Public, Atlas-Full had higher matched point estimates for Bias-ID accuracy (+4.2 percentage points), contamination-free items (+8.4 points), the strict fidelity composite (+8.9 points), and the soft composite (+7.4 points). The reported confidence intervals included zero, and none of the 12 primary comparisons survived Holm correction. Effects were heterogeneous: for Confirmation Bias, identification increased descriptively from 10.7% under Public to 75.9% under Atlas-DJ and 65.4% under Atlas-Full; most Public-condition misidentifications were assigned Bandwagon Effect or Authority Bias. These results support a conditional, package-level interpretation of expert grounding and an iterative workflow combining generation, blind review, and confusion analysis, while leaving downstream model utility and broader generalization for future study.