acceptodds
Under review as a conference paper at ICLR 2027

EviPatrol: Evidence-Grounded Split-Modal MLLM for Audio-Visual Deepfake Localization and Diagnosis

Abstract

Audio-visual deepfake localization requires identifying the manipulated modality, precise temporal boundaries, and supporting evidence. However, existing methods often retain localization-related cues implicitly within latent representations or prediction scores, limiting their interpretability and downstream reasoning. Meanwhile, multimodal large language models (MLLMs) struggle to capture weak and localized boundary signals from raw audio-visual inputs due to their emphasis on semantic understanding. We propose EviPatrol, an explicit boundary evidence modeling framework for fine-grained audio-visual deepfake localization and diagnosis. EviPatrol transforms implicit localization cues into structured, time-indexed evidence representations through a Boundary Evidence Interface before MLLM reasoning. To improve evidence reliability, we introduce BLA-AMR and Safe-AMR to enhance modality-aware evidence learning and reduce cross-modal shortcuts. The resulting evidence is integrated with audio-visual content through a split-modal MLLM for independent audio and visual localization with evidence-grounded diagnosis. Extensive experiments on LAV-DF and AV-Deepfake1M demonstrate that EviPatrol achieves superior temporal localization and cross-dataset generalization. On LAV-DF, EviPatrol improves [email protected] by 51.94 percentage points over the previous best method, validating the effectiveness of explicit boundary evidence modeling under strict localization criteria.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Audio-Visual Deepfake Localization, Multimodal Large Language Models, Temporal Forgery Localization, Explicit Boundary Evidence, Evidence-Grounded Reasoning, Multimodal Forensics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7jD5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces a new MLLM-based audio-visual method for detecting and localizing deepfake manipulations. The method is equipped with auxiliary components (the representation extractors, shift-alignment network and Boundary-AEP module) that provide explicit evidences extracted from audio and visual streams that are further used by the MLLM to predict evidence-grounded decisions and diagnosis. Furthermore, the authors introduce two key components: BLA-AMR, which regulates the learning process of the Boundary-AEP, balancing the contribution of each modality and Safe-AMR, which is responsible for alleviating the reliance on cross-modal shortcuts (audio dataset-specific or audio-visual disagreement shortcuts). The evidence is used by a split-modal MLLM trained using modality-specific LoRA that analyzes it together with the input audio-visual sample to produce localization predictions and corresponding explanations. The authors evaluate their method against a set of deepfake temporal forgery localization methods on two widely used datasets: LAV-DF and AV-Deepfake1M, achieving state-of-the-art performance.

Critical strengths
  1. This work addresses two important issues:
  • inconsistency methods are limited to detecting audio-visual disagreements, yielding predicted regions that cannot be categorized as visual-only, audio-only or audio-visual manipulated since this information is missing. EviPatrol has specialized modules that regulates the model such that it is able to correctly predict visual-only, audio-only and audio-visual manipulated regions.
  • exiting approaches learn implicit forensic cues in the latent space through supervised learning, transforming them into black boxes. As a consequence, classic methods can provide localized predictions but without a clear evidence (especially no textual explanations). This raises interpretability concerns, a common issue of deepfake detection. On the other hand, EviPatrol is able to predict a full diagnosis of the detection decision.
  1. The method's performance is supported by experimental results, achieving substantial improvements over state-of-the-art methods especially under a strict localization setup (AP\@0.95AP\@0.95). Moreover the ablations prove the capability of the full framework.

  2. While some information is missing, the work is generally well structured, clear and easy to follow. The contributions are clearly stated and the motivation is well justified.

Critical weaknesses

Anchors:

  • The granularity of anchors can determine the detection’s quality. As such, the choice of 0.5 seconds/ a maximum of 32 anchors should be ablated.
  • What happens when a video is substantially larger than samples from AV-Deepfake1M/LAV-DF (especially for in-the-wild data which tend to contain significantly longer videos)? The anchors' granularity decreases which might lead to a quality degradation.

Boundary-AEP:

  • This component contains a wide range of modules. While theoretical justifications are provided for these choices, a more granular ablation is required to confirm empirically the necessity of each module.
  • It is not clear on what data is the reconstruction module from Boundary-AEP trained on. Is it trained on real-data only or also on manipulated data? For the latter, the reconstruction module will learn to reconstruct manipulated regions as well as normal regions, which defeats the purpose of this module.

Safe-AMR:

  • For the paired-state discrepancies computation, do the two corresponding videos contain the same semantic input? Otherwise the difference between the two normalized decision losses do not reflect a discrepancy when audio is manipulated, but rather different sample difficulties. The authors should specify this.
  • Why is Safe-AMR evaluating only in one direction? Visual evidence that influences audio decisions is also possible in theory. The authors should provide evidence that supports their hypothesis.
  • In the main paper the formulae for Δ+,t\Delta_{+,t} and Δ−,t\Delta_{-,t} are presented. Then, in appendix D.2 the authors replace the computation with Δt\Delta_t. Furthermore, the tolerance margin is no longer present in the final Δt\Delta_t formula. This is a writing consistency issue that makes the Safe-AMR methodology hard to follow and should be addressed.
  • the value of NprefN_{pref} is not specified and it is highly dependent on the output formatting.

Evaluation:

  • Authors should specify what baselines are retrained/finetuned and for what baselines the trained checkpoints were utilized.
  • It is unclear if the video-only methods are evaluated against the video-only groundtruth, specifically on AV-Deepfake1M. If the methods are evaluated on the union ground-truth, then the evaluation is unfair. If not, the evaluation is not comparable. The authors should specify this.
  • A comparison with another MLLM-based temporal forgery localization method would have improved the baseline comparison greatly. For example, using Omni-Fake-R1, a method already cited by the authors.

Minor:

  • mDPO is wrongly cited.
Questions for authors
  • Lm,0coreL^{core}_{m, 0} is computed on training or validation splits? On a mini-batch or averaged across the full split?
  • What happens if the split between two anchors is done during a phoneme or viseme? Wouldn’t this lead to a quality degradation?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 7jD5

Dear Reviewer,

Thank you for your comments. We address the main points below.

W1 / Q2. Anchor resolution, long videos, and phoneme boundaries. Anchors index evidence on a nominal 0.5-second grid; the MLLM predicts continuous timestamps rather than anchor-quantized boundaries. When more than 32 anchors are needed, Appendix B.1 uses 32 duration-adaptive windows covering the full timeline without truncation. Wider windows may weaken brief-anomaly representation; full coverage does not establish robustness at arbitrary video lengths.

Dense features are extracted before pooling. Intra-anchor transitions and cross-anchor temporal modeling retain context, so a cut within a phoneme or viseme does not directly discard that information.

W3. Semantic matching in Safe-AMR. Safe-AMR uses two kinds of pairing. RF/FF and RR/FR compare state groups with the same visual label but different audio states; they do not require semantically matched source content. Their loss gap is a training-monitoring signal, not by itself a causal effect of changing audio.

The normal and audio-withheld branches, in contrast, form a same-example intervention pair: the visual input and visual target remain fixed while the audio condition changes. This avoids the content and difficulty differences inherent in comparing two different videos. Safe-AMR uses both state-group statistics and same-example decision sensitivity, rather than relying only on the former.

W5 / Q1. Formula consistency and training statistics. The appendix equation omits the margin from the main-text definition. The correct expression averages the two margin-thresholded discrepancies:

Δt=12(Δ+,t+Δ−,t)\Delta_t = \frac{1}{2}(\Delta_{+,t} + \Delta_{-,t})

The tolerance margin affects the shortcut score and counterfactual sampling probability through these discrepancies. It does not appear again in the final loss.

These statistics are updated online from training-batch observations, synchronized across devices, using state-specific EMAs. They do not use validation data or full-split recomputation. Controls are read before the current observations update the controller.

W6. Ground-truth alignment and comparability. V-AP and U-AP measure different tasks. A like-for-like V-AP comparison uses visual ground truth for both video-only baselines and EviPatrol. A U-AP comparison uses union ground truth for every method. A video-only result on the union task is a restricted-input reference, not an input-matched audio-visual comparison.

W7. Comparison with Omni-Fake-R1. Its public audio-visual talking-head branch predicts global authenticity rather than modality-specific temporal intervals. Other branches provide audio temporal localization and video spatial localization. The public evaluator reports IoU/F1, not temporal AP.

Its published results therefore cannot be inserted directly into our same-protocol comparison. A task-adapted baseline would require extending audio-visual temporal outputs and adapting supervision, rewards, and evaluation.

Minor. mDPO citation. The correct reference is Wang et al. (2024), mDPO: Conditional Preference Optimization for Multimodal Large Language Models.

Dear authors,

Thank you for your response. While some of the concerns were (partially) addressed, I would have appreciated more empirical proofs (e.g. performance degradation based on video lengths, ablation on Boundary-AEP components, comparison against MLLM methods) in order to further prove the applicability and usefulness of EviPatrol. Furthermore, several concerns are still not addressed. As a result, I will keep my Weak Rejection rating.

Reviewer_8HLvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses audio-visual deepfake localization: determining whether audio, video, or both have been manipulated, identifying the affected time intervals, and providing supporting evidence. To help MLLMs detect subtle, localized forensic cues, EviPatrol introduces a Boundary Evidence Interface that organizes audio, visual, and cross-modal temporal signals into time-indexed Evidence Cards. It combines frozen MARLIN and Whisper encoders with evidence prediction and modality-aware training components, then uses separate Qwen2.5-Omni-7B LoRA branches to localize audio and visual manipulations. Before generating a diagnosis, the system freezes its localization results; the diagnostic module can explain a predicted interval using cited evidence but cannot alter the predicted modality, timestamps, confidence, or interval boundaries. Experiments on benchmark and cross-dataset settings show that EviPatrol performs strongly in deepfake localization, while its evidence-grounded explanations are rated as more understandable, trustworthy, and verifiable than prediction-only or unconstrained MLLM explanations.

Critical strengths
  1. The paper goes beyond simply detecting whether a video is fake. It also identifies the manipulated modality, locates the affected time intervals, and provides supporting evidence. These outputs are useful for forensic analysis, where knowing what was changed and where it occurred matters.
  2. The Boundary Evidence Interface links forensic features with the MLLM’s reasoning. Evidence extraction, interval localization, and explanation generation are handled as separate steps. This makes the process easier to inspect.
  3. The LAV-DF results show a notable improvement at the strict [email protected] threshold. The ablation study also suggests that Boundary-AEP, BLA-AMR, Safe-AMR, and the MLLM localizer each contribute to the overall performance.
Critical weaknesses
  1. The system combines pretrained encoders, boundary-evidence prediction, modality rebalancing, and shortcut reduction. These components may account for much of the localization performance. The MLLM may mainly turn the extracted evidence into text. A comparison with a conventional localization model using the same evidence would help show what the MLLM adds.
  2. The method aims to locate audio and visual manipulations separately. But if the benchmark uses the same time intervals for both modalities, it is hard to tell whether the model can identify modality-specific boundaries. The paper should clarify this limitation and, if possible, evaluate on data with separate audio and visual annotations.
  3. Prior work has already studied audio-visual deepfake detection, modality-specific features, and cross-modal consistency. EviPatrol’s main addition seems to be using an MLLM to organize evidence and produce diagnostic explanations. The paper should clarify how this differs from earlier detection and representation-learning methods.
Questions for authors
  1. Could you evaluate audio and visual localization using separate time-span annotations? LAV-DF assigns the same 'fake_periods' to both manipulated modalities. How, then, do A-mAP and V-mAP show that the model can locate each modality’s boundaries separately?
  2. How do you verify that the diagnostic text is factually grounded? Anchor checks confirm structural consistency, but do they show that each observation is supported by the audio or video?
  3. Does Safe-AMR reduce cross-modal shortcuts? Could you test visual localization after removing or replacing the audio, and audio localization after removing or replacing the video? Please report how these changes affect both detection and localization.
1 comment
AuthorsOpenReview ↗

Response to Reviewer 8HLv

Dear Reviewer,

Thank you for your comments. We address the main points below.

W1. The MLLM predicts modality-specific intervals from the original audio-video input and Evidence, rather than merely converting evidence into text. Boundary-AEP produces intermediate evidence; diagnostic explanations are generated only after localization is frozen. In Table 3, with BLA-AMR and Safe-AMR disabled, Boundary-AEP-only and Boundary-AEP+MLLM achieve U-mAP of 21.74 and 63.25, respectively.

W2/Q1. The two branches are trained independently on LAV-DF and predict audio and visual intervals without sharing predicted timestamps. The union is constructed after branch-level prediction. A-mAP and V-mAP are evaluated against the corresponding modality annotations. Shared fake_periods in FF cases reflect the annotation definition, not a constraint tying the outputs together. AV1M test annotations are not publicly available; test evaluation must use the official evaluation channel.

W3. Our contribution is explicit boundary-evidence modeling before prediction, not a textual explanation added afterward. Evidence Cards organize abnormality, boundary likelihood, and interval support into semantically defined, time-indexed evidence. The split-modal MLLM localizes manipulations using the original media and evidence. Diagnosis explains frozen predictions without changing their intervals, modalities, or localization AP.

Q2. Timestamps, modalities, anchors, and evidence fields are assembled deterministically from the frozen results; the model generates observations and judgments. The verifier checks citations, temporal overlap, and modality consistency. Establishing factual support for an observation requires checking the original media. Table 7 measures how explanations help users understand and verify predictions; structural validity is not equated with factual accuracy.

Q3. Bidirectional modality intervention. The table reports target-modality localization mAP (%) when the other modality is removed or replaced.

Target modalityOriginal inputOther modality removedOther modality replaced
Visual (V-mAP)78.8667.6468.87
Audio (A-mAP)87.1173.9675.31
Reviewer_PBuqOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EviPatrol which is a staged pipeline for audio-visual temporal forgery localization (TFL). A trainable “Boundary-AEP” model on frozen MARLIN and Whisper-large-v3 features gets per-anchor scores (0.5 s anchors, capped at 32). These cover abnormality, onset and offset proximity, interval support, AV inconsistency and lag. The scores are rendered into a text “Evidence Pack.” Two Qwen2.5-Omni-7B LoRA adapters are used, one for the audio and one for the visual, to take the raw clip plus the Evidence Pack and give JSON intervals. The union is a deterministic merge of the two outputs. Two training-only regularizers are added: BLA-AMR (loss reweighting based on per-modality learning progress and gradient dominance) and Safe-AMR (an audio-withheld counterfactual branch for the visual adapter). A third adapter then generates a post-hoc explanation, which passes through a structural verifier.

Critical strengths
  • For each modality, there are dedicated trainable parameters but still using the full AV context. This makes it a fair design.

  • The Safe-AMR counterfactual (withhold audio, work on the visual target) is an appropriate way to probe audio-to-visual shortcutting.

Critical weaknesses
  1. Table 1 mixes metrics computed against different ground truths. The Audio-Head (87.11) and Visual-Head (78.86) "Average" values equal the A-mAP and V-mAP in Table 3, row 6. Per Appendix A.2, these are scored against derived modality-specific labels, not the official union fake_periods. An audio-only branch scored on the union GT could not reach ~89 [email protected], because it would miss every visual-only fake. These rows sit in the same column as baselines evaluated on the union protocol, and the caption does not say so. Move them to a separate table, or evaluate every row the same way.

  2. The SOTA claim and the baseline selection are not accurate. The abstract and Sec. 4.2 call DeformTrace (34.23 [email protected]) the previous best on LAV-DF. That ignores several stronger published results.

    • UMMAFormer, which the paper cites and includes in Table 2 but omits from Table 1, reports 98.83 / 95.54 / 37.61 AP@{0.5, 0.75, 0.95} on the LAV-DF full set. That beats EviPatrol's 97.98 at [email protected], which contradicts the bolding in Table 1.
    • Later work reports much higher strict-IoU numbers. UniSkip-Mamba lists a previous best [email protected] of 53.6% from Uni-CaCLF and reports 63.4% itself. UniSkip-Mamba may be concurrent, but Uni-CaCLF is not.
    • A survey notes that CFPRF raised LAV-DF [email protected] to 88.61%. I believe that number comes from an audio-focused setting, but the authors should address it explicitly.

    The headline "+51.94 pp over the previous best" is therefore not supportable as written, and Table 1 needs to be redone with the current literature.

  3. Which split are the results on? Appendix A opens with "Reporting the size of an official test set does not imply that the test set was accessed." It also says development-set results use the full dev split. Read together, this suggests some main-table numbers may be dev-split numbers placed next to baselines reported on test. Please state explicitly, for Tables 1, 2, 3, 5, and 6, which split each number is computed on. For AV1M, also explain how test-set labels were obtained (public release, or the challenge evaluation server). If any EviPatrol results are on dev while baselines are on test, the comparison is invalid.

  4. The strict-IoU result needs a mechanistic explanation and sanity checks. [email protected] on sub-second segments requires boundary errors of a few tens of milliseconds. The evidence grid is 0.5 s (coarser for clips over 16 s). Qwen2.5-Omni ingests video at a low frame rate and emits timestamps as text. It is not clear where 20–40 ms precision comes from. Please report:

    • the video frame rate and audio settings fed to Qwen2.5-Omni;
    • a histogram of boundary errors in milliseconds;
    • [email protected] stratified by segment duration;
    • a simple "snap to Whisper/forced-alignment word boundary" baseline.

    LAV-DF and AV1M manipulations are word-level, so an ASR-pretrained audio encoder plus an LLM may be recovering annotation-aligned word boundaries rather than detecting forgery boundaries. That would still be useful, but it is a different claim.

  5. Confounded attribution. The method differs from the baselines in at least four ways at once: Whisper-large-v3 + MARLIN features, a 7B backbone, two separate adapters, and the evidence interface. Table 3 row 1 (Boundary-AEP alone, U-mAP 21.74) does not explain how intervals are decoded without the MLLM. A properly decoded detection head should do much better than that. The key missing baseline is ActionFormer, UMMAFormer, or a similar head trained on the same MARLIN+Whisper features (or on Boundary-AEP outputs) with standard NMS. Without it, I cannot tell whether the gains come from "explicit evidence + MLLM reasoning" or from stronger features and capacity.

  6. The ablations are internally inconsistent.

    • Safe-AMR is described as applied only to the visual adapter, with the audio adapter "independently trained" (D.2, Sec. 4.1). Yet in Table 3, adding Safe-AMR raises A-mAP from 72.92→78.97 (row 3→5) and from 80.23→87.11 (row 4→6). Please explain how a visual-adapter-only regularizer changes audio-branch AP.
    • Row 5 vs. row 4: A-mAP and V-mAP move by about 1–2 points, but U-mAP jumps from 74.28→83.78. The union is a deterministic merge of the two branches. Was the confidence-only scorer φ_prop on in some rows and off in others? The paper never says whether φ_prop was "selected" for the reported results.
  7. Safe-AMR regularizes a token the evaluation ignores. The "decision prefix" (Eq. 71) is the start of the answer, i.e., the s field. Appendix C.2 states that s is not used for the evaluated state, since decisions come from segments only. D.2 also says the target places the visual decision "before the long diagnostic suffix," while C.2 says the localization stage produces no diagnostic prose. Which target format was actually used? And why should regularizing s transfer to interval emission?

  8. Table 5 does not measure what it claims. Four-state accuracy mixes audio-branch and visual-branch errors. The shortcut hypothesis is about whether the visual decision varies with audio state. The direct test is visual-branch accuracy on RF vs. FF and on RR vs. FR, ideally also under audio-withheld inference. The numbers as given do not clearly support the text either. The FR column moves the most, yet FR is an audio-fake/visual-real state, so its accuracy depends heavily on the audio branch.

  9. BLA-AMR novelty and positioning. Reweighting by per-modality learning progress and gradient dominance is close to the modality-imbalance literature: OGM-GE (Peng et al., CVPR 2022), PMR, AGM, greedy-learner analyses, and others. None of these are cited or compared against. The related-work section on modality bias cites only "mDPO" via Zhang et al. 2024, but that reference is the video-LMM DPO paper, not mDPO (Wang et al., 2024). The controller also has more than 15 hand-set hyperparameters (δ, δ_g, ρ_dom, τ_dom, g_min, ρ_lp, τ_lp, w, β, κ_j, µ_max, B_max, λ_Δ, λ_C, p_min, p_max, …). None of their values are reported, even though Sec. 4.1 says they are in the appendix.

  10. The diagnosis component is thinly evaluated.

    • The source of training targets for the explanation adapter ∆_exp is never specified (human-written, LLM-generated, or templated). This matters a great deal for interpreting Table 7.
    • The verifier checks only structure (anchor existence, overlap, modality, decision). The paper acknowledges that it does not check whether observations like "facial boundary becomes temporally unstable" are true. The illustrative explanations are generic enough to fit almost any positive.
    • The human study lacks basic details: number of clips, participant expertise, which MLLM and prompt produced the "MLLM Explanation" baseline, whether conditions were blinded and counterbalanced, inter-rater agreement, significance tests, and the abstention rate. The ethics statement mentions consent but not IRB/ethics approval.
  11. Cross-dataset (Table 6). It is unclear whether the baselines were retrained by the authors under the stated protocol or taken from elsewhere. DeformTrace, the strongest in-domain baseline, is missing. Since this result appears in the abstract, it should be in the main paper with protocol details.

Minor

  • Figure 1 and Figure 2 use what appear to be licensed cartoon characters. Please replace them with original or neutral artwork.
  • Eq. 7 (main text: two discrepancies with margin δ_s) and Eq. 72 (appendix: a single averaged discrepancy with no margin) disagree.
  • Sec. 2.3 cites "Omni-Fake-R1 (Li et al., 2026)," but the reference is a benchmark paper with a different title.
  • Table formatting is inconsistent (12.8, 83.4, 08.14, leading zeros).
  • The LAV-DF per-modality GT derivation (copying fake_periods to every manipulated modality) is an assumption. Since LAV-DF fakes are temporally co-located across modalities, the split-modal design is not really tested on asynchronous AV manipulations there. This should be said explicitly.
  • The appendix spends a lot of space on audit and protocol language while omitting hyperparameters and training-data details for the explanation stage. The balance should be reversed.
Questions for authors
  1. Which split is every number in Tables 1–3 and 5–6 computed on? How were AV1M test labels accessed?
  2. Can you provide Table 1 with all EviPatrol rows evaluated on the official union GT, and with UMMAFormer and the more recent LAV-DF methods included?
  3. What are the boundary-error distribution and the duration-stratified [email protected]? What frame rate is given to Qwen2.5-Omni?
  4. How does an ActionFormer/UMMAFormer head on identical MARLIN+Whisper features perform? How was row 1 of Table 3 decoded?
  5. Why does Safe-AMR change audio-branch mAP? Was φ_prop used, and in which rows?
  6. Where do the explanation training targets come from?
  7. What are the BLA-AMR and Safe-AMR hyperparameter values, and how sensitive are results to them?
2 comments
AuthorsOpenReview ↗

Response to Reviewer PBuq

Dear Reviewer,

Thank you for your comments. We address the main points below.

W1 / Q2. Evaluation in Table 1. Both the Audio Head and Visual Head receive the full audio-video input, but each predicts manipulation intervals for its own modality and is scored against the corresponding ground truth. Thus, 89.43 for the Audio Head is audio [email protected], not union AP. The two head results are modality-specific diagnostics and do not enter the union AP ranking. For jointly manipulated LAV-DF clips, the branches share interval annotations, but their predictions are not constrained to have identical timestamps.

W2. Baselines and state-of-the-art claims. UMMAFormer is a relevant LAV-DF baseline. UniCaCLF and UniSkip-Mamba report [email protected] of 53.61 and 63.4, respectively. The 51.94 percentage-point difference is relative to the DeformTrace result listed in Table 1, not to the best published result. CFPRF reports 88.64 for audio forgery localization, which is not part of the union AP ranking.

W3 / Q1. AV-Deepfake1M test scoring. AV-Deepfake1M test annotations are not public. For official test evaluation, researchers submit predictions to the 1M-Deepfakes Challenge on Codabench, which scores them against hidden ground truth.

W4-W5. Boundary precision and MLLM localization. The 0.5-second windows organize Evidence Cards; they do not set the resolution of predicted endpoints. The localizer uses the original audio and video together with onset and offset cues in the Evidence Pack to predict continuous timestamps that need not lie on the anchor grid. In Table 3, with BLA-AMR and Safe-AMR both disabled, adding the evidence-grounded MLLM localizer raises U-mAP from 21.74 to 63.25, a gain of 41.51 percentage points.

W6-W8. Safe-AMR and audio reliance. In our setup, Qwen2.5-Omni tends to rely on audio cues. Safe-AMR targets overreliance on audio in the visual branch by withholding audio and related evidence for selected training samples while keeping the video and visual target fixed. The audio branch was also retrained for the corresponding configurations in Table 3, so the A-mAP changes are not a direct effect of applying Safe-AMR to the audio branch.

Removing or replacing audio reduces V-mAP from 78.86 to 67.64 or 68.87, respectively. Removing or replacing video reduces A-mAP from 87.11 to 73.96 or 75.31, respectively. The branches predict intervals for different modalities, but both benefit from the full audio-video context.

Reviewer_PBuqOpenReview ↗

Reviewer response

I thank the authors for their response.

The key distinction is that the authors explain why their design might work but have not reported the experiments needed to show whether the proposed components work as claimed.

I would not require every minor presentation issue to be resolved, but the missing experiments around [email protected], SOTA comparisons and methodological attribution are substantial enough. Hence, I would maintain Weak Reject and request that the authors consider the weaknesses for a stronger submission.

Comments on the paper

1 comment

Dear Reviewers,

The authors have responded to your questions. If you haven’t already done so, please review their responses and let them know whether your concerns have been addressed. Please use the discussion to raise any remaining concerns or follow-up questions.

Thank you for your time!

Best,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.