acceptodds
Under review as a conference paper at ICLR 2027

Expert-Grounded Synthetic Data for Cognitive Biases: Controlled Generation and Blind Evaluation of Construct Fidelity

Abstract

Reliable systems for identifying cognitive-bias mechanisms require datasets whose labels are psychologically faithful, domain-grounded, and able to distinguish neighboring constructs. Large language models can scale candidate generation, but generic prompting may produce fluent items that express a different mechanism from the intended target, creating mechanism-level label noise. We study this problem using a private, psychologist-curated Cognitive Bias Domain Atlas containing 229 cognitive biases, 30 application domains, and 2,385 curated bias–domain relationships whose records may include expert definitions, domain-specific justifications, and reference examples. We compare three conditioning packages over 31 matched relationships spanning 10 biases and five domains: Public, using a literature-grounded definition; Atlas-DJ, using an expert definition and domain-specific justification; and Atlas-Full, which additionally supplies an expert reference example with no-copy and overlap safeguards. Across 93 generation cells, the system produced 930 candidate survey items, of which 857 were eligible for condition and target-blind review by a psychology expert. Primary inference is conducted at the relationship level. Relative to Public, Atlas-Full had higher matched point estimates for Bias-ID accuracy (+4.2 percentage points), contamination-free items (+8.4 points), the strict fidelity composite (+8.9 points), and the soft composite (+7.4 points). The reported confidence intervals included zero, and none of the 12 primary comparisons survived Holm correction. Effects were heterogeneous: for Confirmation Bias, identification increased descriptively from 10.7% under Public to 75.9% under Atlas-DJ and 65.4% under Atlas-Full; most Public-condition misidentifications were assigned Bandwagon Effect or Authority Bias. These results support a conditional, package-level interpretation of expert grounding and an iterative workflow combining generation, blind review, and confusion analysis, while leaving downstream model utility and broader generalization for future study.

Primary area:
applications to neuroscience & cognitive science
Keywords:
synthetic data generation, cognitive biases, construct validity, data-centric machine learning, dataset curation, human evaluation, large language models, psychometrics

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_H9QtOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper focuses on the synthetic generation of cognitive biases (e.g., confirmation bias) with GPT-4o-2024-08-06, given a predefined generation taxonomy containing 10 cognitive biases, 5 domains, and 31 bias–domain couplings. This taxonomy was taken from a larger, private taxonomy curated by a trained psychologist.

The authors vary how detailed the information given to the generating model is:

  1. Public uses literature-based definitions of the biases.
  2. Atlas-DJ contains the psychologist's definitions of the biases along with domain justifications.
  3. Atlas-Full, the most detailed, additionally contains a reference example and instructs the generating model not to copy or overlap with it during generation.

The same psychologist who designed the original taxonomy evaluates the generated data blind for construct fidelity, using the following metrics:

  1. bias identification (primary outcome),
  2. neighboring-bias contamination (primary outcome),
  3. domain relevance (secondary/descriptive outcome),
  4. usability (secondary/descriptive outcome),
  5. editorial decision (secondary/descriptive outcome), and
  6. a loose or strict composite of the preceding metrics (primary outcome).

At a high level, the authors find that Atlas-Full improves over Public on contamination-free generation (by 8.4 pp), on the strict fidelity composite (by 8.9 pp), and on the soft composite (by 7.4 pp). However, the authors report that these contrasts remain uncertain.

Critical strengths

Sound metrics and statistical analysis. The chosen metrics are intuitive and appropriate. The sampling and statistical analysis are careful and sufficient to back the key claims made. The answers to the RQs are sufficiently corroborated by the findings and analysis.

Critical weaknesses
  1. Clarity of the abstract and introduction. The abstract and introduction introduce numerous terms that cannot be interpreted until much later in the paper. For instance, the abstract mentions "neighboring constructs," "generation cells," and "no-copy and overlap safeguards" without clarifying them. Similarly, "zero confidence intervals" and "Holm correction" appear without any discussion of their implications, which makes them hard to interpret. Paragraphs 3 and 4 of the introduction repeat the same information in the abstract but offer no additional context to interpret it clearly. Much of this information remains unclear until later in the paper, leaving the reader hanging. The paper becomes much clearer from the related work section onward. It would be great if the authors carefully revisited this formative text so that it is self-contained and interpretable at a high level, giving a good foundation for interpreting the rest of the paper. It would help to introduce terms and jargon only when there is enough context to interpret them.

  2. Scope, novelty, and impact.

  3. The premise of the work focuses specifically on construct fidelity in the context of cognitive bias generation. This is a rather narrow focus, especially since the authors do not discuss how it may generalize to the construct fidelity of synthetic data more broadly.

  4. The authors study the generation properties of a single model, GPT-4o, so it is unclear whether their findings on construct fidelity apply beyond this model.

  5. The takeaways are the principles embodied in Atlas-Full. If we were to synthesize what their value to the community could be:

    ◦ (a) examples help improve construct fidelity (it is well known in the field that in-context examples improve synthesis quality); ◦ (b) clear definitions help with generation quality (which is intuitive); and ◦ (c) expert definitions of cognitive biases improve over public/literature definitions (which applies only narrowly, to types of data whose public and expert definitions differ).

Altogether, although the empirical investigation is sound and apt, the narrow focus of the premise, the single-model setting, and the applicability and impact of the findings for the community account for my recommended decision. Finally, the authors' stated impact of this work is to motivate synthesizing cognitive data in an iterative data-development loop. This appears intuitive and is already widely adopted in domains beyond cognitive biases.

Questions for authors
  1. Why is cognitive bias data an appropriate setting for studying the construct fidelity of synthetic data?
  2. Concretely, who in the ICLR/academic community does this work benefit, and how?
  3. Are there generalizable construct-fidelity takeaways for the ML community invested in generating synthetic data? If so, how do they differ from existing data synthesis practices?
Reviewer_ipUpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks whether expert knowledge helps a language model generate items that express the intended cognitive bias. It compares public definitions, expert definitions with domain explanations, and a fuller package containing reference examples. Blinded evaluation measures construct identification, neighboring construct contamination, and usability. The aggregate trends favor richer conditioning, but uncertainty remains large. Confirmation Bias accounts for the clearest identification improvement.

Critical strengths

s1. The evaluation targets the right problem. A fluent item can still carry the wrong label. Measuring construct identification and confusion with neighboring mechanisms gets closer to the reliability of the synthetic data than a general quality score would.

s2. The statistical treatment is appropriately cautious. Items generated for the same relationship are not treated as independent evidence, and the authors account for multiple comparisons. I appreciate that the conclusions do not simply follow the ordering of the point estimates.

s3. The Confirmation Bias analysis is informative. Errors fall mainly into Bandwagon Effect and Authority Bias, suggesting a specific conceptual confusion. The authors also show how much the aggregate advantage changes when Confirmation Bias is removed. That transparency helps.

Critical weaknesses

w1. The central evaluation depends on one reviewer who helped curate the expert resource. Blinding the condition is useful, but it cannot remove shared assumptions about where construct boundaries lie. Would an independent expert make the same distinctions on the disputed items? Repeated ratings from the original reviewer cannot answer that question.

w2. Several factors change together across conditions. The fullest package also adds an extra filtering procedure. The authors acknowledge that these are package comparisons, but the source of improvement remains unclear. Reporting successful output relative to all generated candidates would help distinguish better generation from a different selection of retained items.

w3. The Confirmation Bias result needs a closer examination of the actual conditioning text. Once this bias is removed, the aggregate identification advantage reverses under item weighting. This makes the public definition unusually consequential. I would like to see matched definitions and representative failures discussed together. Otherwise, the strongest finding could partly reflect a weakness in this particular baseline wording.

w4. Some examples leave me unsure whether the intended mechanism is actually identifiable. Mentioning a stock’s historical purchase price before asking for a valuation does not, by itself, establish anchoring. What response pattern would distinguish anchoring from reasonable use of historical information? The paper needs a clearer boundary between recognizing a bias cue in text and establishing that an item measures that bias.

Questions for authors

See above comments. The main quality measure rests on an evaluator closely connected to the conditioning resource, and the strongest result remains difficult to separate from the wording of the public baseline.

Reviewer_mfG8OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper studies whether ChatGPT-4o can generate valid examples of human cognitive biases, and whether prompting it with particular instructions improves this generation. They choose 31 pairs of biases and settings to evaluate and use a trained psychologist to provide ground truth labels for the bias labels. They then used this dataset to evaluate different prompts for creating psychometric data.

Critical strengths

S1. Used a trained psychologist to label the data rather than relying on general vendors.

S2. Revealed that prompting does matter for the accuracy of the model at generating data with the correct label.

Critical weaknesses

W1. I'm not sure what problem this is solving or why it's surprising that prompting is important for generating accurate data. In general I don't see a strong novel contribution.

W2. Only one model was tested, so it's unclear how much this generalizes at all.

W3. Only one labeler was used so the data could be somewhat skewed by that.

W4. It seems that most of the results in table 7 are not statistically significant and in table 12 if you remove the "confirmation bias" results, the accuracy gain is essentially 0 or slightly negative.

Questions for authors

I don't have any questions.

Reviewer_E3wnOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies whether Large language models (LLMs) can be used to write the examples of cognitive bias. Having a huge collection of cognitive bias can be useful to train and test computer systems to locate or mitigate the bias. Human written bias may be time consuming for large scale training or evaluation. The paper studies biases such as confirmation bias, bandwagon bias, authority bias, etc. The authors find that LLMs can synthesize the bias that sounds reasonable on the surface but shows the wrong bias. The authors evaluated a total of 10 out of 229 biases. An LLM was used to write bias in three setups: Public: in this setup, only the definition of bias was given Atlas-DJ: in this setup, definition and explanation of how these biases show in real world Atlas-Full: in this setup, definition, explanation as well as examples were provided. A psychology expert assessed all the generated examples of biases without being told which bias was expected to be exhibited. It was found that atlas-full setup was better among all cases.

Critical strengths

This paper studies a weakness of LLM in writing examples of cognitive bias. It shows LLMs may not be good at capturing the nuances between similar biases. The paper reveals that GPT-4o isn't particularly good, especially at confirmation bias.

Critical weaknesses

The results depend on just one psychology expert, which indicates a lack of rigor and depth of the results. If the same expert who wrote the examples and definition reviewed the LLM's response, then they might be just mimicking the expert's way of reasoning / thinking. Multiple reviews from independent experts should be performed. Were strategies such as a few-shot prompt , self-reflection, etc used? The baselines are not enough for ICLR's standard. Why is the LLM especially bad at confirmation bias ? How is this bias different from other bias that makes LLM bad at it? It may be due to poorly written definitions. Only 10 out of 229 biases were chosen and studied. Why were those specific 10 biases chosen? Overall, the comparison results are not significant. The tested model was only one, that too GPT-4o which is not one of current leading LLMs. This seems to be a fairly shallow evaluation, which makes it hard for this paper to be recommended for acceptance.

Questions for authors

None.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.