VeriLoop: Multimodal Fact Verification as an Agentic Closed-Loop Sequential Decision Process
Abstract
Evidence-based multimodal fact verification has advanced retrieval, evidence structuring, and reasoning, but a verifier must also decide what remains worth investigating and when to stop. We formulate multimodal fact verification as a closed-loop sequential decision problem and introduce VeriLoop, which maintains explicit proposition-level verification state, selects from that state what to investigate next, acquires targeted textual, visual, or cross-modal evidence, updates and replans, and stops; local proposition verification is separated from deterministic claim-level verdict derivation, so every verdict is traceable and the process itself becomes measurable. On Factify, MOCHEG, and VERITE, controlled ablations show proposition-level resolution interacts strongly with evidence quality: with graph, evidence store, and backbone fixed, it changes macro-F1 by and under benchmark-supplied evidence but by under noisy retrieval, and chain-of-thought does not reproduce the effect. Reconstructing 26,417 logged interventions shows verification utility is front-loaded: first interventions dominate, most episodes end on budget, and an offline stopping-policy counterfactual shows stopping policy, verdict composition, and label granularity are coupled. Multimodal fact verification should be treated not only as reasoning over evidence, but as controlling a sequential verification process.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.