- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces OmniVL-Guard Pro, a tool-augmented agent built on Qwen3VL-8B. It handles forgery detection, grounding, and segmentation across text, image, video, and cross-modal inputs, plus real-time event verification. The agent can call web search, cropping, zooming, edge-anomaly screening, face detection, frame extraction, and SAM3. The paper contributes (1) the open-world agent itself, which calls web search, cropping, zooming, edge-anomaly screening, face detection, frame extraction, and SAM3; (2) FSTR, a dataset of tool-use trajectories built by a tree-structured, self-evolving generation pipeline; and (3) Checker-Guided Agentic RL, in which a trained Checker scores whether the reasoning process supports the answer and scales the reward of correct responses accordingly. The authors report gains over prior methods on in-domain and five OOD benchmarks.
Critical strengths
- The problem formulation and the CGARL idea. Moving forensics from closed-world prediction to active, tool-based evidence seeking is practically meaningful, and CGARL explicitly separates final-answer correctness from whether the tool trajectory actually supports the answer.
- Broad evaluation with some process-level evidence. The paper covers five modalities, a new real-time verification benchmark, five OOD benchmarks, stage-wise and tool-wise ablations, and a human evaluation in which pseudo-success falls from 25.4% to 7.2%.
Critical weaknesses
- The central claim that CGARL produces evidence-grounded reasoning is not established. (a) The ablation is confounded: the full model in Table 3 receives an extra RL stage on the additional 30% of data in plus an extra reward, compared with "SFT + ARSPO"; without an outcome-only Stage III control on the same data and steps, the gain from 21.64 to 27.88 cannot be attributed to the Checker. (b) The paper's own case studies show the failure mode CGARL is meant to remove. In Case 3, the first zoom call uses exactly the ground-truth box [455, 195, 575, 449] before any evidence is gathered; the subsequent top edge-anomaly box [529, 500, 611, 625] does not overlap it, yet the reasoning claims it "overlaps significantly", and the answer is the initial zoom box. Case 4 (video) and Case 5 (painting) also contain an identical tool call and response (zoom_image on [520, 320, 620, 420], returning [520, 320, 619, 420]) on unrelated inputs. This is consequential because process consistency is the paper's key differentiator from outcome-only tool RL; without it, the remaining contribution is largely an engineering combination of tool use, process supervision, and a data pipeline.
- The headline gains are measured against weak or construction-suppressed baselines. The +30.71 on RealFact is computed against tool-free Qwen3VL-235B 's 55.74, but RealFact was filtered with Qwen3VL-235B's parametric memory (As you state at Appendix B.1), so this baseline is suppressed by design. Against tool-using baselines in the appendix, the margin is +5.86 over GPT5.4 ReAct and +6.00 over DEFAME; the Text-Video gain drops from +26.96 to +2.76 against FakeSV-VLM. Moreover, the RealFact split does not state whether claims from the same event are kept in one split, and evaluation uses live web retrieval. This is consequential because the "state-of-the-art" claim reduces to modest margins on a benchmark whose leakage and reproducibility are unverified.
Questions for authors
- Can you add a control in Table 3 that, after Stage I, continues outcome-only RL on for the same number of steps as full CGARL, with and without ? This is needed to attribute the gain from 21.64 to 27.88 to the Checker.
- For the human evaluation in Appendix F.3, how were trajectories sampled, how many annotators judged each one, and were they blinded to model identity and Checker scores? What exactly does the 96.2% agreement measure, and is the pseudo-success rate computed over all trajectories or only correct ones?
- In Case 3, how did the model choose the exact ground-truth box as its first zoom region without prior evidence, and why does the reasoning claim overlap with an edge-anomaly box that does not overlap it?
- Why do Case 4 and Case 5 contain an identical zoom_image call and response on unrelated inputs? Are the case-study trajectories raw model outputs, and can you show randomly sampled ones, including failures?
- How are RealFact examples partitioned across SFT, RL, and testing, and are claims derived from the same underlying event kept in the same split?
- Can you report RealFact results on a fixed evidence snapshot, and with fact-checking pages or other sources that directly state the verdict excluded, to test whether the model reasons over evidence or copies a retrieved conclusion?