VERIBENCH-WILD: A BENCHMARK AND MEASUREMENT TESTBED FOR MISINFORMATION EVALUATION

We introduce VERIBENCH-WILD, a benchmark and measurement testbed for studying how evaluation choices shape misinformation benchmark scores. A higher score need not mean better detection; it also depends on the information a model receives, its source and adaptation setting, and how its outputs are scored. VERIBENCH-WILD provides 27,158 multilingual but English-dominant social-media records with frozen splits, a primary mechanism-typology task, five further task views, public and full representations, and reproducible reference baselines. Across five evaluated models, removing the fact-check note reduces macro-F1 by 12.7–21.3 points, while note-only inputs remain close to full-input performance. The cross-source gap changes substantially with the adaptation regime, and restricting training and evaluation to common-support examples reduces but does not eliminate it. For generated claims, eight automatic evaluation pipelines, each a judge with its parser, accept the same outputs at rates from 15.3% to 63.4%. These choices change rankings, not only scores: a wrong fact-check note erases the lead of API models over open-weight models, QLoRA fine-tuning beats in-context learning within source but loses to it across sources, and ranking judges by acceptance nearly reverses their ranking by agreement with three blinded human raters. Across all eight judges, the parser matters only when a judge ignores the requested format, which moves one judge from 0.4% to 31.9%. Scores on \veribench should therefore state their information inputs, source and adaptation settings, and evaluator and parser configuration, so that comparisons are interpreted under explicit conditions.