acceptodds
Withdrawn from ICLR 2027

Learning What to Transfer: Permission-Routed Mechanism Experts for Multimodal Short-Video Misinformation Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Detecting misinformation in short videos requires reasoning over both internal multimodal content and external event evidence. Existing methods often fuse heterogeneous internal cues into a shared representation or directly incorporate retrieved precedents, overlooking differences among manipulation mechanisms and the transferability of external information. To address this issue, we propose Permission-Routed Mechanism Experts (PRME). PRME assigns soft Strong, Weak, and No permissions to retrieved precedents: Strong permits factual feature transfer and mechanism guidance, Weak retains only mechanism-level priors, and No relies solely on internal evidence. A shared pool of mechanism-specific experts then performs specialized authenticity reasoning. Experiments on FakeSV and FakeTT demonstrate that PRME consistently improves retrieval-augmented misinformation detection, achieving notable gains on the FakeSV temporal and event splits and the FakeTT temporal split. Ablation and relation-stratified analyses further validate the effectiveness of permission-aware evidence transfer and mechanism-specific expert routing.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal misinformation detection, retrieval-augmented learning, multimodal representation learning, mixture-of-experts, evidence routing

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_MpNqOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes PRME for multimodal short-video misinformation detection. It assigns retrieved evidence three soft permissions: factual and mechanism transfer, mechanism guidance only, or reliance on internal evidence. The branches share eight mechanism experts and a general expert. Experiments on FakeSV and FakeTT cover temporal and event splits, with gains reported on both temporal splits and the FakeSV event split.

Critical strengths
  1. The distinction between factual transfer and mechanism guidance provides a useful motivation for handling retrieved evidence with varying relevance.
  2. The evaluation covers two datasets, two split protocols, and a broad set of architectural controls, allowing the benefits and limitations of the design to be examined.
Critical weaknesses
  1. The fairness of the main comparison remains unclear. Most earlier baseline scores are taken from TrueLens, while TrueLens itself is reproduced at lower performance. The authors should clarify any protocol differences that could explain this drop.
  2. The paper does not convincingly establish a substantial methodological contribution. Although the ablations support several design choices, they do not establish that the experts develop their intended mechanism specializations or that permission routing correctly identifies which factual information is transferable.
Questions for authors
  1. What explains the gap between the original and reproduced TrueLens results? If the reproduction uses a different experimental protocol, are the borrowed baseline scores still comparable, or should those methods also be evaluated under the same settings?
  2. Beyond aggregate classification improvements, what evidence shows that the experts specialize in their named mechanisms and that the router’s permissions correspond to actual factual transferability?
Reviewer_XbptOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies retrieval-augmented short-video misinformation detection, focusing on the risk of transferring unreliable factual information from retrieved precedents and the limitations of generic multimodal reasoning. The authors propose Permission-Routed Mechanism Experts (PRME), which assigns Strong, Weak, or No permissions to control whether retrieved evidence can provide factual features, mechanism-level guidance, or no external information. It further introduces mechanism-specific experts to support specialized reasoning about different misinformation patterns.

Critical strengths
  1. The paper clearly identifies the risks of uncontrolled evidence transfer and the limitations of generic multimodal reasoning in misinformation detection.

  2. The proposed framework and its permission-routing mechanism are presented in a clear and understandable manner.

Critical weaknesses
  1. The rationale for selecting the eight manipulation mechanisms is not sufficiently explained. A clearer justification of this taxonomy, including its coverage of common misinformation patterns and the distinction between categories, would better support the design of the mechanism-specific expert pool.

  2. Although the mechanism-aware gating improves detection performance, the paper does not directly demonstrate that individual experts learn their intended manipulation mechanisms. An analysis of expert activation patterns across different mechanism categories would help substantiate the claimed specialization.

  3. Experiments are conducted on FakeSV and FakeTT. Evaluation on an additional benchmark, such as FakeVV, with an appropriate retrieval protocol would help assess whether PRME generalizes to different types of misinformation and manipulation patterns.

Questions for authors
  1. What motivated the selection of these eight manipulation mechanisms, and how do they cover common misinformation patterns without substantial overlap?

  2. Can the authors provide evidence that individual experts specialize in their intended mechanisms, for example, through expert activation patterns across different mechanism categories?

  3. Could the authors clarify how PRME would generalize to additional benchmarks such as FakeVV, given potential differences in manipulation types and annotation schemes?

Convincing responses would strengthen my assessment and potentially increase my rating.

Reviewer_NQc6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses a practical problem in retrieval-augmented short-video misinformation detection: a semantically similar precedent may not provide transferable event facts, but may still offer useful cues about manipulation mechanisms. PRME uses PCS/TSS retrieval-consistency scores to assign soft Strong, Weak, and No permissions to three branches. Strong allows interaction with the top-ranked precedent's features and incorporates a mechanism prior; Weak preserves the query's internal representation and incorporates only an aggregated mechanism prior; No receives no precedent-derived inputs. The branches share eight mechanism experts and one general expert, and their authenticity logits are combined using the soft permission weights. Training additionally uses mechanism supervision and load balancing. Relative to the reproduced TrueLens baseline, PRME improves Macro-F1 by 0.75 and 2.54 percentage points on the FakeSV temporal and event-isolated splits, and by 2.04 and 0.08 points on the corresponding FakeTT splits. Capacity-matched, no-external-prior, and mechanism-agnostic MoE controls support the value of the overall design on temporal splits. On event-isolated splits, some simpler configurations match or exceed Full PRME's mean classification performance, while Full performs better on several calibration metrics. The evidence therefore supports a retrieval-context-dependent method that considers both classification and confidence quality, rather than a general retrieval strategy with a clear advantage across every split.

Critical strengths

S1. A well-motivated problem with meaningful information-access constraints Separating whether facts can be transferred from whether mechanism cues can be borrowed is more specific than weighting evidence using a single retrieval-similarity score. Figure 1 effectively illustrates why directly reusing facts from a topically similar but different event can produce invalid analogies. Equations (8)–(14) also implement genuine differences in branch-level input access: Weak and No do not directly inject precedent features. These are understandable structural distinctions, not merely three names for the same computational path.

S2. Substantial positive evidence on temporal splits Table 2 and Appendices D/E go beyond comparing the final model against existing baselines: they include controls for capacity, expert organization, external priors, and individual branches. These results support the usefulness of external information and mechanism guidance in the evaluated temporal settings, particularly on FakeTT. The overall gains cannot simply be attributed to adding classifier heads or parameters. However, these controls do not fully isolate the contribution of mechanism identity itself, as discussed in W1.

S3. Calibration results add value beyond mean classification scores In Table D.11, random hard routing achieves a mean Macro-F1 of 91.168 on the FakeSV event split, slightly above Full's 90.968. However, its ECE, Brier score, and NLL are 0.067, 0.079, and 0.408, respectively, all worse than Full's 0.051, 0.074, and 0.311. This is practically relevant: a retrieval policy may affect not only classification accuracy but also overconfidence induced by unreliable evidence. The high-similarity, different-event subset in Table D.13 is also informative. PRME does not outperform TrueLens on classification metrics, but reduces ECE from 0.1082 to 0.0608. The authors appropriately distinguish this result from improved classification robustness.

S4. Good transparency about evaluation protocols and negative results The paper states that each split uses a training-only precedent bank, removes query self-matches, and inserts neither test samples nor test-derived representations. It reports five temporal seeds, five event-isolated folds, paired tests, and multiple-comparison analyses. Appendix E explicitly acknowledges that PRME is effectively tied with TrueLens on the FakeTT event split and that some simpler models have higher means, rather than presenting only favorable configurations. This transparency increases confidence in the work.

Critical weaknesses

W1. The contributions of mechanism identity and precedent class composition/prior density are not sufficiently disentangled (Major) Equation (19) gives authentic precedents all-ones affinities and fake precedents smoothed mechanism labels. These priors may therefore encode class composition and density as well as mechanism identity, affecting mechanism-versus-general-expert weighting. Existing mechanism-agnostic and Authentic target 0 controls do not isolate these effects. A targeted control should retain internal supervision while replacing external priors with mean-matched density vectors or randomizing mechanism identities while preserving labels, counts, and marginal frequencies. Supervision of the mechanism scorer also does not establish functional specialization of the authenticity heads. This is an attribution concern, not evidence of test-label leakage or technical invalidity.

W2. The metric definitions and values in Table B.6 require a prioritized consistency check (Major) Table B.6 reports Overall ACC/M-F1 of 0.9347/0.9422 and Same-topic values of 0.8780/0.9093. Under Equation (C.1), standard binary Macro-F1 cannot exceed Accuracy for identical predictions, including equally weighted averages across runs. B.6 also reports TrueLens M-F1 of 94.22%, versus 93.35% in the main table. Please reconcile metric definitions, run provenance, and aggregation using per-run confusion matrices and metric code. This may be a local reporting issue rather than evidence against the main experiments; verifying those results would substantially reduce this concern.

W3. Event-isolated evidence supports conservative fallback, but the permission policy's independent classification gains remain limited (Moderate) On FakeSV event splits, Full M-F1 (90.968) is close to internal-only (90.666) and No-only (90.852); its 2.54-point gain over TrueLens cannot be attributed entirely to routing. On FakeTT event splits, Full achieves 76.939 ± 5.218 versus 78.543 ± 4.304 for the matched deep MLP. These means establish neither significance nor equivalence. Limited classification gains do not refute conservative fallback when transferable precedents are scarce, and calibration benefits remain valuable. The main claims should distinguish temporal-split gains, event-isolated fallback, and probability quality.

W4. The actual object of soft permission and the reproducible implementation need clarification (Moderate) Strong permission uses maximum candidate PCS, but feature injection uses Top-1 under a different retrieval score; these may identify different precedents. Please report their agreement and whether the injected item satisfies the intended consistency criterion. C.3 should also clarify frozen PCS/TSS generation, the event coefficient, which router parameters are trained, and how event consistency is obtained without dataset identifiers. Soft routing itself is not a flaw: 85.75% argmax-No usage does not imply zero external contribution. Average soft weights and residual Strong contributions for Weak/No cases would clarify the actual degree of suppression.

Questions for authors

Q1. Regarding W1: What happens if internal mechanism supervision is kept fixed and only the stored precedent priors are replaced with density-matched vectors or randomized mechanism identities? Can this control show that the gains are not merely due to precedent class composition or general-expert gating? Is there expert-intervention evidence on mechanism-matched examples supporting functional specialization of the classifier heads?

Q2. Regarding W2: Could the authors provide the run provenance, per-run confusion matrices, and metric implementation for Table B.6, and explain why Macro-F1 exceeds Accuracy under the definition in Equation (C.1)? Do the 94.22 in B.6 and the main-table TrueLens value of 93.35 come from different checkpoints, predictions, or aggregation procedures?

Q3. Regarding W3: Under event isolation, is the main intended claim improved classification, improved calibration, or transferring reliable precedents while avoiding negative transfer when none are available? Could classification and calibration differences between Full and No-only/internal-only be reported separately using paired results from the same folds or seeds? A large collection of additional broad baselines is not necessary.

Q4. Regarding W4: How often is the maximum-PCS candidate that determines Strong permission the same as the top-ranked item actually injected? How is the default frozen retrieval artifact generated, what is the event-term coefficient, and which routing parameters are trained? Are dataset event identifiers used at inference, and what configuration applies when they are unavailable?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.