Multimodal Bias in AI-Mediated Markets: Benchmarking, Preference Optimization, and Human Valuation

Generative models mediate economic decisions by selecting the information people see. Demographic associations in that selection process can change consumer valuations even when the underlying product and reviews remain fixed. We investigate this pathway through multimodal benchmarking, paired review-summary generation, Direct Preference Optimization (DPO), and an incentivized human marketplace experiment. In a multimodal benchmark adapted from a standard human implicit association test (IAT), the unadapted instruction-tuned Gemma 3 12B checkpoint (baseline Gemma) exhibits implicit demographic bias in negative-descriptor selection. Across 839 paired review sets, Black-cue review summaries score 12.6% lower in sentiment relative to the White-cue mean under the study’s generation procedure. DPO fine-tuning with low-rank adaptation uses 1,000 preference pairs. The reported sentiment gap narrows from −1.061 to −0.007 score points. An ethics-approved, preregistered, incentive-compatible marketplace study includes 976 eligible completions and 15,616 product evaluations. Under randomized assignment of review summaries and seller photos, review summaries from baseline Gemma yield an estimated Black-cue versus White-cue willingness-to-pay gap of −18.7% in the study’s reference condition, compared with −0.6% for summaries from DPO-tuned Gemma. The estimated willingness-to-pay gap therefore narrows by 18.1 percentage points (95% CI: 4.4, 31.8; p = .0097). The findings connect measurable model behavior to human–AI interaction: generated market information changes incentivized human valuation, and DPO-tuned review summaries yield a smaller downstream valuation gap.