EVIDENCE LEDGERS: AUDITABLE CLAIM-LEVEL GROUNDING FOR VISION–LANGUAGE MODELS

Vision–language model answers often contain several factual claims, but standard generation provides no reliable record of the input evidence that supports each claim. We introduce evidence ledgers, which align evidence atoms, typed claims, and answer units through two routes, a direct evidence-to-answer alignment and one composed through the claims, with a null source that absorbs claim mass not assigned to the available evidence. Training combines a claim-level unlikelihood term weighted by the null allocation, a counterfactual swap loss, and consistency between the two routes on the model’s own drafts; inference emits a claim-level certificate whose SUPPORTED verdict is calibrated by learn-then-test to control the false-support rate among SUPPORTED claims, provided that calibration and deployment claims follow the same distribution. On Qwen3-VL-235B the strict, human-audited unsupported-claim rate falls from 11.0% to 4.6%. The corresponding rates are 7.2% for on-policy grounder-DPO and 7.0% for GRPO trained with the same grounding signal at matched compute, and 6.8% for grounder-DPO with tool outputs in context; HallusionBench accuracy rises by 10.4 points, and the gains persist on long-form hallucination benchmarks and relabelled POPE and replicate on InternVL3.5-38B. Controlled comparisons assess the contributions of the loss terms: the unlikelihood term contributes most, the swap and alignment terms add smaller increments, and coupling claims through shared evidence capacity adds under one point overall. In a study using identical answers, annotators reading ledger certificates needed 19.7% less verification time than those reading grounder certificates.