acceptodds
Under review as a conference paper at ICLR 2027

ForgeLoc-R1: Set-Level Reinforcement Fine-Tuning of Video MLLMs for Temporal Forgery Localization

Abstract

Temporal forgery localization predicts every manipulated interval in a partially forged video. Existing methods rely on detectors built for particular manipulation traces, whereas a general-purpose video multimodal large language model (MLLM) could localize any forgery family. Reinforcement fine-tuning with GRPO and an overlap reward is the standard recipe for temporal grounding, but transplanted to forgery localization it loses its learning signal. We show that both losses are inherent in the group-relative update: a group whose sampled intervals miss or barely overlap a short segment yields rewards that differ only by sampling noise, and the update is biased toward the segment count most frequent in the training data. ForgeLoc-R1restores the signal at both points while leaving the GRPO update unchanged, and scores each rollout with a set reward that the two mechanisms act on. 1)At the group level, anchor-guided advantage pruning inserts the ground-truth answer into every group as an anchor, which we show gives a group without signal a likelihood step toward the ground truth. Reward shaping then bounds the anchor's margin, and only the anchor and the rollouts with the largest advantage magnitudes are trained on. 2)At the reward level, a set-level reward counts the distinct segments a rollout has found, so a missed segment and an unmatched prediction both lower it, and ranks the misses by their distance to the ground truth. 3)At the objective level, a count-balanced objective weights multi-segment videos so that the policy is not biased toward the most frequent segment count. On ActivityForensics and DDL, ForgeLoc-R1 outperforms the strongest reinforcement fine-tuning baseline by 4.34 and 3.11 mIoU, with the largest gains on multi-segment videos and on a generator held out from training.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Temporal Forgery Localization, MLLM-based Video Understanding, Reinforcement Fine-Tuning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nzRTOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ForgeLoc-R1, a reinforcement fine-tuning framework for temporal forgery localization using video multimodal large language models. It combines Anchor-Guided Advantage Pruning, a Set-Level Reward, and a Count-Balanced Objective to address weak learning signals and segment-count bias in GRPO. Experiments on ActivityForensics and DDL show improvements over reinforcement fine-tuning baselines, particularly for multi-segment localization and an unseen video generator.

Critical strengths
  • Important topic.
  • Well-motivated method design.
  • Controlled comparisons across two benchmarks.
Critical weaknesses
  • The novelty over existing anchor-guided and rollout-selection methods is insufficiently established, with several important baselines missing.

  • Some experimental conclusions are inconsistent with the reported results, and the ablations do not fully isolate individual contributions.

  • Limited statistical validation, missing code, and insufficient evaluation under realistic deployment conditions weaken the reliability and generality of the findings.

Questions for authors
  1. The novelty of ForgeLoc-R1 requires stronger differentiation from existing reinforcement fine-tuning methods. TempSamp-R1 already injects ground-truth solutions into GRPO groups, while [1] incorporates off-policy guidance into policy optimization. Moreover, the advantage-based rollout selection resembles the down-sampling strategy in [2], and the count-balanced objective adopts repeat-factor sampling from LVIS [3]. What are the fundamental technical contributions beyond combining these existing mechanisms for temporal forgery localization? In particular, how is the importance ratio handled for a ground-truth anchor that was not sampled from the behavior policy? Since Proposition 1 effectively introduces a weighted likelihood update toward the ground-truth answer, comparisons with SFT and SFT-then-GRPO under matched training budgets are also necessary to establish the benefits of the proposed reinforcement learning design.

  2. The experimental comparisons are insufficient to establish the practical advantage of using a general-purpose MLLM for temporal forgery localization. Although the introduction argues that dedicated detectors are limited to specific manipulation traces, no dedicated temporal forgery localizer is evaluated. Could the authors compare ForgeLoc-R1 against representative detectors such as UMMAFormer and Feng et al. (2026b) under the same benchmark splits? The related work should also discuss "VidGuard-R1" [4], "FakeShield" [5], and the closely related "VidForensics-M1" [6], which investigates reinforcement learning with verifiable temporal grounding for video forensics. These comparisons would better establish both the novelty and practical competitiveness of the proposed approach.

  3. Several experimental interpretations appear inconsistent with the reported results. Section 4.1 attributes the DDL improvements primarily to localizing more segments rather than obtaining tighter boundaries, but Table 5 shows that ForgeLoc-R1 matches fewer DDL segments than TempSamp-R1 (73.5% versus 76.1%) while achieving smaller boundary errors. Table 3 also reports a higher under-reporting rate on DDL (65.0% versus 58.3%), contradicting the explanation that the method consistently recovers more manipulated segments. Furthermore, Figure 2(a) shows almost no improvement over TempSamp-R1 for ActivityForensics segments shorter than three seconds, despite the emphasis on short-segment optimization. Could the authors reconcile these observations and clarify whether the gains mainly arise from improved segment coverage, boundary precision, or both?

  4. The ablation studies and statistical validation require improvement. Table 2 removes multiple components simultaneously: removing the set-level reward also removes the count weight, while removing Anchor-Guided Advantage Pruning eliminates both the ground-truth anchor and advantage pruning. Consequently, the individual contributions of these mechanisms remain unclear. Could the authors provide separate ablations for anchor insertion, reward shaping, advantage pruning, set-level reward, and count balancing? The ablations should also be evaluated on DDL, where the segment characteristics differ substantially from ActivityForensics. In addition, the reported improvements are based on results without confidence intervals or repeated-run statistics. Reporting means and standard deviations across independent seeds would strengthen the conclusions.

  5. The paper does not provide implementation code, substantially limiting reproducibility. ForgeLoc-R1 involves several implementation-sensitive components, including ground-truth anchor insertion, reward shaping, advantage selection, Sinkhorn-based matching, and count-balanced sampling. Important hyperparameters are also insufficiently specified, including the reward-shaping parameters in Eq. 5 and the GRPO optimization settings. Moreover, ForgeLoc-R1 generates seven sampled rollouts and retains four training responses per group, but the corresponding rollout counts and computational budgets of the baselines are not clearly reported. Please also provide the wall-clock training costs and clarify whether the comparisons use matched generated-rollout budgets rather than only matched optimizer settings.

  6. The theoretical analysis and evaluation scope require stronger justification. The paper describes ForgeLoc-R1 as theoretically grounded, but several propositions rely on simplified assumptions, and the advantage-pruning rule is explicitly described as a heuristic that does not inherit the corresponding theoretical guarantee. Direct measurements of reward variance and the frequency of uninformative GRPO groups would better support the proposed mechanism. Furthermore, the evaluation considers only forged videos, and the prompt explicitly assumes that at least one manipulated segment exists, leaving false-positive behavior on authentic videos unexplored. Could the authors evaluate mixed authentic-forged test sets and report false-alarm rates? Generalization is also assessed using only one held-out generator, which is insufficient to establish robustness across diverse unseen manipulation methods.

  7. Several methodological, presentation, and bibliographic details require correction. The timestamp-aware IoU from "Time-R1" is used without a corresponding reference. The Cuturi (2013) citation has an incorrect paper title (should be "Sinkhorn Distances: Lightspeed Computation of Optimal Transport"), and several entries retain outdated preprint information despite formal publication. Additionally, multiple mathematical symbols are reused for different quantities, including the reward-shaping and KL coefficients, which makes the derivations harder to follow. Figure 1 contains minor labeling errors ("Hig h"), the delta row of Table 1 mixes one decimal for ActivityForensics with two decimals for the DDL mIoU column, and the qualitative examples primarily highlight successful cases without showing representative failures.

[1] Yan, Jianhao, et al. "Learning to Reason under Off-Policy Guidance." NeurIPS. 2026.

[2] Xu, Yixuan Even, et al. "Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learning." TMLR. 2026.

[3] Gupta, Agrim, Piotr Dollar, and Ross Girshick. "LVIS: A Dataset for Large Vocabulary Instance Segmentation." CVPR. 2019.

[4] Park, Kyoungjun, et al. "VidGuard-R1: AI-Generated Video Detection and Explanation via Reasoning MLLMs and RL." ICLR. 2026.

[5] Xu, Zhipei, et al. "FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models." ICLR. 2025.

[6] Liu, Bowei, et al. "VidForensics-M1: Meta-Detection Reinforcement Learning with Verifiable Temporal Grounding for AI-Generated Video Forensics." arXiv preprint arXiv:2608.11201. 2026.

Reviewer_n6ByOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper fine-tunes a video MLLM to locate all manipulated intervals in a video. It combines a ground-truth anchor, advantage-based rollout pruning, a set-level reward and higher training weight for multi-segment videos. On ActivityForensics and DDL, it improves mIoU over TempSamp-R1. The paper also evaluates segment counts, boundary errors and an ood generator.

Critical strengths
  1. This paper proposes several effective solutions to the challenges involved in localizing multiple forgeries within a video. The mIoU and F1 of ForgeLoc-R1 improves on all datasets.
Critical weaknesses
  1. The connection between the reward analysis and the implementation is not fully established. Eq. 7 and Proposition 2 use exact one-to-one matching to explain how the reward handles missed segments and extra predictions, while training uses a soft matching formulation that is not generally equivalent. The paper does not clearly show whether the implemented reward preserves the intended ranking of incomplete, duplicate, or overly long predictions.
Questions for authors

Could you clarify whether the soft matching reward used in training preserves the behavior of the exact one-to-one reward in Eq. 7? A few examples involving a missed segment, duplicate predictions, or one long interval covering two ground-truth segments would help.

Reviewer_b9N4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ForgeLoc-R1, which combines ground-truth-anchor-guided advantage pruning, a set-level reward, and count-balanced training to reinforcement-fine-tune a video MLLM for temporal forgery localization. On ActivityForensics and DDL, the method achieves better overall localization performance than the evaluated reinforcement fine-tuning baselines.

Critical strengths

1.Clearly defined research problem. The paper addresses sparse reward signals and segment-count bias in temporal forgery localization through training strategies tailored to these challenges.

2.Improved overall performance. On ActivityForensics and DDL, the method outperforms the evaluated reinforcement fine-tuning baselines in mIoU and F1 at multiple thresholds, with additional evaluation on an unseen generator.

Critical weaknesses

1.Some experimental results conflict with the claims in the introduction. The introduction emphasizes correcting segment-count bias and reducing under-reporting on multi-segment videos. However, on DDL, the under-reporting rate increases from 58.3% for TempSamp-R1 to 65.0%, while the segment matching rate decreases from 76.1% to 73.5%. The authors should analyze these deteriorations because they conflict with the corresponding claims in the introduction. The paper currently lacks sufficient discussion of this issue.

2.The ablation study lacks independent validation. When evaluating the contribution of the set-level reward, the authors also remove the count weighting. These two components should be evaluated separately.

Questions for authors

Why does ForgeLoc-R1 increase the under-reporting rate and decrease the segment matching rate on DDL compared with TempSamp-R1? How do these results reconcile with the introduction’s claims about reducing segment-count bias and missed segments?

Could the authors provide separate ablations for the set-level reward and count weighting, changing only one component at a time to isolate their contributions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.