acceptodds
Withdrawn from ICLR 2027

VeriLoop: Multimodal Fact Verification as an Agentic Closed-Loop Sequential Decision Process

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Evidence-based multimodal fact verification has advanced retrieval, evidence structuring, and reasoning, but a verifier must also decide what remains worth investigating and when to stop. We formulate multimodal fact verification as a closed-loop sequential decision problem and introduce VeriLoop, which maintains explicit proposition-level verification state, selects from that state what to investigate next, acquires targeted textual, visual, or cross-modal evidence, updates and replans, and stops; local proposition verification is separated from deterministic claim-level verdict derivation, so every verdict is traceable and the process itself becomes measurable. On Factify, MOCHEG, and VERITE, controlled ablations show proposition-level resolution interacts strongly with evidence quality: with graph, evidence store, and backbone fixed, it changes macro-F1 by and under benchmark-supplied evidence but by under noisy retrieval, and chain-of-thought does not reproduce the effect. Reconstructing 26,417 logged interventions shows verification utility is front-loaded: first interventions dominate, most episodes end on budget, and an offline stopping-policy counterfactual shows stopping policy, verdict composition, and label granularity are coupled. Multimodal fact verification should be treated not only as reasoning over evidence, but as controlling a sequential verification process.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Fact Verification, Agentic AI, Closed-Loop Verification, Sequential Decision Making, Multimodal Reasoning, Evidence Retrieval, Misinformation Detection

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QwZPOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper models multimodal fact verification as a closed-loop sequential decision problem for an agent and proposes the VeriLoop framework. The framework builds a proposition-level verification graph, continuously maintains an explicit verification state, autonomously chooses propositions to verify, gathers text/image cross-modal evidence, updates the state, and decides when to stop; it separates local verification of single propositions from deterministic assertion-based conclusion derivation, ensuring conclusions are traceable. Experiments on the Factify, MOCHEG, and VERITE datasets show that the effectiveness of proposition decomposition verification highly depends on evidence quality—clean evidence improves performance, while noisy retrieval actually harms it, and this effect cannot be reproduced using chain-of-thought reasoning. Verification gains are concentrated in the early steps, and more verification actions do not necessarily lead to better results; stopping strategies, conclusion combination rules, and label granularity are interdependent. The paper points out that multimodal fact verification is not just about reasoning based on evidence but also about controlling the entire sequential verification process.

Critical strengths
  1. Novel problem formulation framing multimodal fact‑verification as closed‑loop sequential decision‑making Most prior work treats fact‑checking as one‑shot or multi‑hop reasoning given evidence. Instead, VeriLoop explicitly models mutable proposition‑level verification state, targeted evidence acquisition, replanning and stopping. Separating local proposition verification from deterministic claim‑level verdict derivation brings full traceability for every final prediction. This agentic control perspective opens a new research direction for fact‑verification.

  2. Empirically rich, careful empirical investigation yielding actionable insights for the community Beyond proposing VeriLoop architecture, the paper systematically studies when structured per‑proposition resolution helps/hurts, diminishing returns of repeated verification attempts, and dangerous benchmark artefacts in Factify / VERITE. The finding “more verification steps ≠ better performance” is counter‑intuitive and well‑supported by logged trajectories and budgeted offline replays; benchmark‑integrity diagnostics serve as important warning for future multimodal fact‑verification benchmark development.

Critical weaknesses
  1. Key stopping‑policy conclusion relies entirely on offline counterfactual truncation without live online validation The promising macro‑F1 improvement under “assertion‑settled stopping” is obtained by cutting pre‑recorded trajectories post‑hoc, not by running the agent live with that stopping condition. As authors note, we cannot confirm whether this stopping rule would generalise to live execution, new datasets or benchmarks supporting the (MISLEADING) label. Without live agent rollouts, the practical utility of this stopping heuristic remains open.

  2. Limited competitive leaderboard performance under zero‑shot setting, strong results depend on gold‑quality evidence VeriLoop obtains strong numbers mainly when gold reference evidence is provided. Under noisy retrieved evidence its per‑proposition resolution mechanism degrades performance (C5 vs C3 in Table2). On VERITE zero‑retrieved‑evidence zero‑shot setting, vanilla VeriLoop underperforms existing fine‑tuned VLMs. While the paper’s main contributions are insights about process control rather than pure leaderboard numbers, readers may expect clearer discussion about real‑world applicability when gold evidence is unavailable.

  3. Poor figure visual quality hinders understanding core results Figures are over‑crowded, lack sufficient annotation, and mix distinct experimental paradigms (live trajectory / offline replay) without prominent visual differentiation. Several key curves and markers are hard to parse. This is a presentation deficit but impedes readers in grasping the paper’s central empirical take‑homes.

Questions for authors
  1. In the assertion‑settled stopping counter‑factual, 473 verdicts change after early truncation. Could you elaborate on qualitative patterns of these changed samples? For instance: how many of these correspond to genuine cases that Factify label‑space cannot capture?

  2. Would it be feasible in future work to run a small‑scale live online experiment with the assertion‑settled stopping rule, to test whether the offline‑observed gain transfers to live agent execution?

  3. On Factify under noisy retrieval, per‑proposition resolution (C5) underperforms flattened reading (C3). Do you have intuitions on what kinds of retrieval errors cause this degradation? Are there simple mitigation directions?

  4. VERITE results show that proposition decomposition helps, but role‑typing hurts performance. Could you give brief intuition why role typing brings negative effect on this benchmark?

  5. Could you substantially revise figures: add clearer legends, explicitly mark which panels correspond to logged live trajectories vs offline frozen replay, and reduce over‑crowding? Many core claims hinge upon these plots.

Reviewer_rtLrOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces VeriLoop, which casts evidence-grounded multimodal fact verification as a sequential control problem. A typed proposition graph records what has been supported, refuted, left unresolved or deemed insufficient. An agent selects a proposition, performs a text, image, or cross-modal intervention, updates the state, may replan, and derives a deterministic claim-level verdict. Experiments on Factify, MOCHEG and VERITE include controlled C3-versus-C5 comparisons (flattened joint reasoning versus per-proposition verification plus rule-based composition), zero-shot and protocol-qualified comparisons with published systems, visual readout analyses, and retrospective analysis of 26,417 logged interventions. Proposition-level composition improves macro-F1 with clean evidence (+9.73 on MOCHEG and +1.64 on Factify) but hurts under noisy Factify retrieval (−9.82). The authors also analyze stopping and prefix trajectories, while explicitly acknowledging that their offline replays do not establish causal controller improvements.

Critical strengths
  1. The motivation addresses a genuine operational issue. A multimodal verifier must determine which aspect of a claim still needs checking and whether more retrieval will change its judgment. Typed proposition states and deterministic verdict composition offer a concrete way to inspect what a verification system believes it has established. This is more substantive than merely adding an extra generation step, even though the novelty of closed-loop control itself requires comparison with existing iterative verification.
  2. The study presents unusually candid, informative negative and diagnostic results. The fixed-graph/evidence/backbone C3–C5 comparisons show that the value of proposition-level verification depends on evidence quality, including a −9.82-point reversal under noisy Factify retrieval. The authors distinguish protocol-incompatible published leaderboard rows, report a Factify label-space mismatch and dataset artifacts, and explicitly separate observed trajectories and offline stopping replay from causal intervention studies. I found no evidence of deliberate baseline suppression or fabricated results in the supplied manuscript.
Critical weaknesses
  1. The main closed-loop contribution is not directly established by the controlled experiments, and its novelty needs a sharper comparison. Section 5’s C3–C5 ablation isolates per-proposition resolution plus deterministic composition, not the effects of state-dependent action selection, replanning, evidence acquisition, or stopping. Factify and VERITE provide logged full-loop runs, but the reported prefix and budget analyses are retrospective; the improved Factify stopping counterfactual truncates frozen trajectories instead of executing a revised policy. Moreover, the authors do not compare the full loop prospectively against a matched fixed-order, no-replanning, or fixed-budget policy. Prior work already employs iterative retrieval/verification with continuation decisions, e.g., FIRE (Findings of NAACL 2025, text-only fact-checking), while REVEAL (FEVER 2026, multimodal verification) conditions later queries on accumulated evidence and allows early stopping. VeriLoop may differ through its persistent typed graph and deterministic composition; the paper should articulate and test that difference rather than treating closed-loop verification broadly as new. These papers use different benchmarks and should not be required as unadapted numerical baselines.
  2. Experimental provenance and comparability remain too weak to independently audit the key controller claims. Appendix K says Factify/VERITE closed-loop runs and the ablation grid were generated from a dirty working tree with no pinned commit, and the MOCHEG C5 evidence-store identity was not captured in its run manifest. Candidate action sets, triggered stop conditions, intervention modalities, and some image evidence were not logged. Replaying saved graph transitions to recover final verdicts checks deterministic derivation, not the policy that generated those transitions. The published comparison also mixes stronger gold evidence with retrieval-based systems on MOCHEG, differing Factify population sizes, and supervised diagnostics with zero-shot systems. The authors flag these limits rather than concealing them, but prominent quantitative wins must not be interpreted as matched evidence for controller superiority.
Questions for authors
  1. Can the authors run the same closed-loop system with matched backbone, evidence, graph, budget, and label mapping against fixed-order verification, no-replanning, and a fixed-budget or fixed-depth stopping rule? Please report both decision quality and inference/retrieval cost. This would establish the value of state-conditioned control beyond the C3–C5 reasoning-mode contrast.
  2. How does VeriLoop technically differ from FIRE’s confidence-based continue/search decision and REVEAL’s evidence-history-conditioned query generation and early termination? Please distinguish their task settings and show which gains specifically require a persistent typed verification graph and deterministic verdict composition. A conceptual comparison is essential even where a matched reimplementation is not feasible.
  3. Can the authors provide a version-pinned runnable implementation, explicit MOCHEG C5 evidence-store checksum, role-specific prompts, and complete per-action traces including selected and rejected candidate actions, modalities and stop triggers? Which central results can an independent group reproduce without relying on authors’ frozen intermediate states?
  4. Can a stopping policy identified by retrospective replay be evaluated prospectively on fresh claims? Please keep the published benchmark comparators separate from the matched controls and quantify the effect of gold-vs-retrieved evidence and label-space adaptations.
Reviewer_6738OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

VeriLoop represents multimodal claims as typed propositions, verifies them individually, and derives the final verdict through deterministic rules. Its controller selects evidence-gathering actions from the evolving graph and decides when to stop. Experiments on Factify, MOCHEG, and VERITE examine evidence quality, visual information, and verification trajectories. The main findings are that proposition-wise verification depends strongly on evidence quality, and stopping behavior interacts with verdict composition and benchmark labels.

Critical strengths
  • Solid Experimentation Across Multiple Benchmarks of Claim Verification
Critical weaknesses
  • Outdated Experimental Setting. The benchmarks were released between 2022 and 2024, and the baselines primarily rely on GPT-4o-mini and Qwen-2-VL. It is unclear whether evaluating multi-model claim verification using outdated models and technical stacks offers meaningful insights for current research. Frontier models may already be capable of solving standard claim verification tasks with ease.
Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.