acceptodds
Withdrawn from ICLR 2027

TruthDebate: Multi-Agent Collaborative Reasoning for Deepfake Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

While multimodal large language models (MLLMs) have made significant progress for deepfake detection, manipulation of deepfake often exhibits various semantically coherent yet deceptive content such as face swap, face emotion editing or text fabrication, making single-view analysis of one MLLM insufficient. LLM-based multi-agent systems (MAS) provide a promising way that collectively mine deceptive content from complementary perspectives to combat this threat. However, conventionally MAS suffer from high computational costs and significant optimization difficulties. To solve these challenges, we propose an adaptive multi-agent framework named TruthDebate for deepfake detection. TruthDebate consists of two core stages: (1) we design a multimodal clue extraction model to capture visual and textual clues of deceptive content to provide prior knowledge as the input of the MAS; (2) Guided by these clues, we design an efficiency-aware multi-agent collaborative reasoning mechanism that includes multimodal clue analysts, evidence integrators, an evidence auditor, and a decision-maker for cooperative reasoning to adaptively optimize MAS topology and routing-based inference. Extensive experiments on the MLLM-Driven synthetic multimodal deepfake dataset demonstrate the effectiveness of our framework. Our framework significantly outperforms multimodal detectors and multi-agent methods while reducing token consumption. Code is available at https://anonymous.4open.science/r/TruthDebate-3131/.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language model-based agents, deepfake detection, multi-agent collaboration

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_9PppOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes TruthDebate, an adaptive multi-agent framework for multimodal deepfake news detection. The framework consists of two stages. First, a multimodal clue extraction model uses learnable clue embeddings and three reasoning branches to predict face swapping, facial emotion editing, and text fabrication. Second, a multi-agent system coordinates specialised analysts, evidence integrators, an evidence auditor, and a decision-maker. The authors introduce feedback-based analyst selection and confidence-based routing to reduce unnecessary agent calls. Experiments are conducted on the MDSM dataset across five news domains, comparing TruthDebate with zero-shot multimodal models, supervised detectors, and multi-agent baselines. The paper reports improvements in detection performance and reduced language-model token consumption, supported by component ablations and additional analyses.

Critical strengths
  1. The paper addresses a practically relevant problem. Detecting semantically coherent multimodal manipulations is challenging, and the attempt to combine visual, textual, and manipulation-specific evidence is meaningful.

  2. The proposed framework has a reasonably clear overall structure, combining multimodal clue extraction with specialised agents, evidence integration, and confidence-based routing. The appendix provides additional mathematical definitions, implementation settings, algorithms, and agent prompts, which help explain the intended workflow.

  3. The authors provide comparisons against supervised multimodal detectors and several multi-agent configurations across five news domains, together with component ablations and token consumption measurements. The reported results show improvements over the evaluated baselines, although the limitations of these comparisons are discussed below.

Critical weaknesses

1. The novelty and technical contribution are insufficient for ICLR 2027.

Applying LLM-based multi-agent collaborative reasoning to fake news detection is no longer a novel research direction. Li et al. [1] introduced FactAgent at ECAI 2024, using an agentic workflow for fact-checking and fake news detection. Lakara et al. [2] investigated multi-agent debate for visual misinformation detection in 2024. More recently, Chen et al. [3] proposed multi-perspective rationale generation, cross-verification, and adaptive evidence weighting for multimodal fake news detection at AAAI 2026. Zhang and Sui [4] also developed an adaptive role-specific multi-agent framework for fake news detection. Wu et al. [5] further explored multi-LLM interaction and collaborative reasoning for fake news detection.

These studies have already explored several key elements of TruthDebate, including specialised reasoning roles, collaborative verification, and adaptive coordination. TruthDebate combines multimodal clue extraction, analyst selection, evidence auditing, and confidence-based routing. However, the paper does not establish what fundamental technical advance this combination provides beyond existing multi-agent detection frameworks. The individual components are largely familiar, and their integration appears incremental. I would have considered this research direction more novel in 2024. For an ICLR 2027 submission, I expect a substantially clearer technical contribution than adapting and combining established multi-agent reasoning techniques for a particular downstream task.

2. The Related Work section is seriously inadequate, raising concerns about the authors' familiarity with the field.

Multimodal fake news and deepfake detection is a well-established and highly active research area. Based on my own examination of the literature, no fewer than 50 relevant papers are published annually in JCR Q1 journals and conferences ranked CCF A/B or CORE A*/A/B. Although the authors are not expected to review every study, the coverage provided in this paper is far from sufficient to reflect the breadth and depth of existing research.

The Related Work section mainly separates multimodal detection from general multi-agent reasoning, without adequately examining the substantial body of research connecting these two directions. It also lacks a critical comparison of existing methods, their limitations, and the specific technical gaps that TruthDebate aims to address. This raises serious concerns about whether the authors have conducted a thorough literature review and are sufficiently familiar with the existing research. It also undermines the credibility of their novelty claims, as the paper does not adequately establish what genuinely distinguishes the proposed approach from previous work.

3. The paper's contribution appears better suited to a more application-oriented conference.

I recognise that ICLR welcomes research across a broad range of machine learning topics. However, I find it difficult to understand why the authors chose to submit this paper to ICLR when conferences such as WWW and WSDM are more directly aligned with fake news detection. The proposed framework mainly applies and combines established multi-agent reasoning techniques for a specific downstream task, with limited methodological novelty.

In my view, an ICLR submission should demonstrate a more substantial technical advance or introduce new ideas with broader relevance to machine learning. I do not see sufficient evidence of either in this paper.

4. The experimental evaluation relies on a single dataset and lacks comparisons on established benchmarks.

The experiments are conducted exclusively on MDSM. Although the authors evaluate five news domains, all of them come from the same synthetic multimodal deepfake dataset. This is insufficient to establish the generalisability of the proposed framework.

The paper does not evaluate TruthDebate on well-established benchmarks such as DGM⁴ [6], Weibo, Twitter, FakeNewsNet (PolitiFact and GossipCop), Fakeddit, NewsCLIPpings, and VERITE. These datasets cover different aspects of multimodal fake news detection and misinformation verification, with established evaluation protocols and competitive baselines. Although these datasets involve different tasks and annotation schemes, the authors could evaluate TruthDebate on those relevant to its detection capabilities. Without such experiments, it is difficult to assess whether TruthDebate is competitive with existing methods on established benchmarks or whether its reported improvements generalise beyond MDSM.

5. The ablation studies do not adequately establish the contribution of the multi-agent reasoning framework.

Figure 2 shows that removing the Multimodal Clue Prior decreases Macro-F1 from 65.33% to 32.40%. However, removing this prior also disables direct prediction and forces all samples through full cooperative reasoning. This changes both the available evidence and the inference procedure.

More importantly, the paper does not report the performance of the trained multimodal clue extraction model alone as a complete detector. Without this comparison, it is difficult to determine how much of TruthDebate's performance comes from the supervised clue extraction stage and how much comes from multi-agent collaboration. The authors should compare the complete method with a prior-only detector, a single-agent method using the same prior and evidence, and simpler aggregation methods under comparable computational budgets. These experiments are necessary to establish whether the additional multi-agent architecture is genuinely responsible for the reported improvements.

6. The reliability estimation and fine-grained prediction procedure need stronger justification.

The proposed feedback-guided reliability estimation is based only on incorrectly predicted manipulation dimensions. As described in Appendix C.2, reliability statistics are updated when the final system prediction is wrong. Consequently, these estimates reflect a role's performance on error-triggered cases, not necessarily its overall reliability. Nevertheless, they are used for analyst selection and evidence weighting. The paper provides no clear evidence that these estimates are well calibrated or more effective than simpler reliability measures computed on validation data.

There is also insufficient explanation of how the three binary manipulation predictions are converted into the six fine-grained classes used for Macro-F1 evaluation, particularly for combined manipulation types. The conversion and final score construction should be specified explicitly.

7. The computational efficiency claims are incomplete.

The paper emphasises reduced token consumption, reporting approximately 1,237 tokens per sample for the complete method. However, Appendix C.4 and E.2 explicitly exclude the Multimodal Clue Extraction Model and precomputed evidence from this calculation. This means that the reported token reduction does not represent the complete computational cost of TruthDebate. The method additionally requires a trained multimodal detector, evidence preparation, multiple specialised agents, and feedback-based analyst selection.

The authors should report end-to-end inference time, computational cost, and the overhead of training and preparing these components. Without these measurements, the practical efficiency advantage of the overall framework remains unclear.

8. Systematic citation formatting errors and incorrect placement of the AI use statement indicate insufficient manuscript proofreading.

The manuscript contains widespread citation formatting errors throughout the Introduction, Related Work, and other sections. For example, the Introduction includes "modalities Zhang et al. (2026)" and "representations Shao et al. (2023; 2024); Ma et al. (2024); Li et al. (2025b)", where the citations should be enclosed in appropriate parentheses. Similar problems occur repeatedly throughout the paper.

Based on my experience, the authors may have used \cite{} where \citep{} was required. These basic LaTeX errors should have been corrected before submission. Their widespread occurrence makes me question whether the authors carefully reviewed the manuscript. Furthermore, the "Use of Generative AI Tools" statement appears after the References as Appendix A, whereas the official ICLR 2027 template places the AI use statement before the bibliography. These issues are straightforward to correct, but their persistence in the submitted manuscript reflects inadequate attention to basic manuscript preparation and quality control.

References

[1] X. Li, Y. Zhang, and E. C. Malthouse, "Large language model agentic approach to fact checking and fake news detection," in Proceedings of the 27th European Conference on Artificial Intelligence (ECAI), 2024.

[2] K. Lakara, G. Channing, J. Sock, C. Rupprecht, P. Torr, J. Collomosse, and C. Schroeder de Witt, "LLM-Consensus: Multi-agent debate for visual misinformation detection," arXiv preprint arXiv:2410.20140, 2024.

[3] J. Chen, Y. Li, K. C. Ng, H. Wang, and L.-J. Zhang, "Toward multimodal fake news detection by multi-perspective rationale generation and verification," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 1, 2026, pp. 66–74.

[4] Z. Zhang and J. Sui, "Deconstruct, diagnose, and deliberate: A protocol-adaptive role-specific multi-agent framework for fake news detection," in Findings of the Association for Computational Linguistics: ACL 2026, 2026, pp. 24516–24534.

[5] L. Wu, X. Jiang, S. Sun, Y. Lei, T. Wen, Y. Wang, and M. Liu, "ZoFia: Zero-shot fake news detection with entity-guided retrieval and multi-LLM interaction," in Findings of the Association for Computational Linguistics: ACL 2026, 2026.

[6] R. Shao, T. Wu, and Z. Liu, "Detecting and grounding multi-modal media manipulation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6904–6913.

Questions for authors
  1. What are the main technical advances of TruthDebate beyond existing agent-based fake news detection and multimodal verification methods, particularly FactAgent [1], LLM-Consensus [2], and the AAAI 2026 work on multi-perspective rationale generation and verification [3]?

  2. Why does the Related Work section omit several directly relevant studies on multi-agent fake news detection? Could the authors provide a more comprehensive review and explain how their work fits within this established research area?

  3. Could the authors evaluate TruthDebate on established independent benchmarks such as DGM⁴, Weibo, Twitter, FakeNewsNet, Fakeddit, NewsCLIPpings, or VERITE, using appropriate competitive baselines? These datasets cover different aspects of multimodal misinformation detection, and results on suitable benchmarks would help establish whether the reported improvements generalise beyond MDSM.

  4. Could the authors report results for the multimodal clue extraction model alone and for a single-agent method using the same prior and evidence? How much improvement can be attributed specifically to multi-agent collaboration?

  5. Why is role reliability estimated only from error-triggered samples? How does this compare with reliability estimates calculated over all validation samples?

  6. How are the three binary manipulation predictions converted into the six fine-grained classes? How are combined manipulation categories and final confidence scores handled?

  7. Could the authors provide end-to-end runtime and computational cost measurements, including multimodal clue extraction and evidence preparation, to support the claimed efficiency improvements?

Reviewer_rvGtOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes TruthDebate, an adaptive multi-agent framework for deepfake news detection. The authors argue that a single MLLM may overlook subtle manipulations, while conventional MAS introduce substantial inference costs and may propagate unreliable evidence. TruthDebate includes two main stages, multimodal clue extraction and multi-agent reasoning. The authors introduce learnable multimodal clue embeddings into a pretrained vision-language encoder and produces a three-dimensional prediction prior corresponding to face swapping, face emotion editing, and text fabrication. The framework uses specialized multimodal clue analysts, visual/textual/global evidence integrators, an evidence auditor, and a final decision-maker. During training, prediction feedback is used to estimate role-specific reliability, generate corrective cues, and select a subset of analysts. The authors report superior performance compared with supervised multimodal detectors, zero-shot MLLMs, and multi-agent baselines.

Critical strengths

The paper targets an increasingly important challenge, which is to detect manipulated multimodal news even though the image and text may appear semantically coherent.

The two-stage architecture is well-motivated. Instead of sending every news sample directly to a computationally expensive group of agents, TruthDebate first obtains structured manipulation clues from a supervised multimodal model.

The paper evaluates on five news-source domains and reports consistent improvements on four unseen domains. The paper evaluates the effects of removing corrective cues, topology optimization, routing, and the multimodal clue prior.

Critical weaknesses
  • The major concern of the proposed method is the novelty and the optimization of MAS. The paper describes a feedback-guided mechanism for optimizing multi-agent topology. However, the actual optimization is relatively simple: the authors estimate agent reliability from training feedback and select the top-k text-related analysts. This may not be a weakness from an engineering perspective. However, it raises concerns about the conceptual novelty and the claimed adaptability.

  • Another problem is whether the reported improvements primarily originate from the supervised multimodal clue extraction model rather than the proposed multi-agent collaborative reasoning. The paper lacks a prior-only baseline that uses the supervised clue extractor to make final predictions without any multi-agent reasoning. Thus, it is unclear to determine the additional benefit from MAS beyond supervised classifier.

  • The paper emphasizes a substantial reduction in inference token consumption. However, the reported online cost explicitly excludes the multimodal clue extraction model and precomputed evidence. While this is a valid measure of incremental LLM token usage, it is not equivalent to the total computational cost of TruthDebate.

  • The evidence auditor is described as an important component that identifies duplicated, missing, or conflicting evidence and adjusts the contribution of each integrator. However, the main component analysis does not include an isolated ablation of the evidence auditor.

Questions for authors
  1. Could the authors provide the performance of the multimodal clue extractor alone, without invoking any agents? How much improvement can be attributed specifically to multi-agent collaborative reasoning?

  2. The proposed topology optimization selects a fixed top-k subset of text-related analysts. How does this differ substantially from conventional reliability-based expert selection, and what makes the resulting topology adaptive at inference time?

  3. Can the authors report end-to-end inference latency and computational cost, including the multimodal clue extractor and evidence preparation, rather than only LLM token consumption?

  4. What is the independent contribution of the evidence auditor? How does the full model compare with an auditor-free implementation or simpler confidence-weighted evidence aggregation?

Reviewer_qJfXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TruthDebate combines a supervised multimodal detector with selectively invoked multi-agent reasoning for multimodal news manipulation detection. The detector produces scores for face swapping, facial emotion editing, and text fabrication. Confident predictions bypass the multi-agent stage; other samples are processed by specialized analysts, evidence integrators, an auditor, and a decision-maker. Training feedback supplies role reliability estimates, corrective cues, and analyst selection. Experiments on MDSM cover one training domain and four unseen news domains, reporting detection performance and online language-model token consumption.

Critical strengths
  1. Meaningful improvements over controlled multi-agent baselines. Tables 1–2 report average Macro-F1 and mAP of 65.33 and 73.81, exceeding the strongest multi-agent baseline for each metric by 6.71 and 5.84 percentage points, respectively. Macro-F1 is also higher than all compared multi-agent baselines on each of the four unseen domains. Since these baselines share the underlying model, prior, and evidence, the comparison provides useful evidence for the effectiveness of the proposed configuration.

  2. An improved performance–cost trade-off within the reported online setting. Compared with disabling all adaptive mechanisms, the full system improves Macro-F1 from 59.74 to 65.33 while reducing average online token consumption from 3768.20 to 1237.25, approximately a 67.2% reduction. This supports the practical value of selectively invoking and organizing the reasoning stage.

Critical weaknesses
  1. The incremental benefit of multi-agent reasoning over the supervised detector and a simpler reasoning pipeline remains unestablished. In Figure 2, removing the clue prior reduces Macro-F1 from 65.33 to 32.40, while disabling routing reduces it to 61.37. These results establish the importance of the supervised prior and selective execution, but do not measure the improvement over the prior detector itself. The paper does not report either a prior-only baseline or a single-agent baseline using the same prior, evidence, and routing rule. The comparisons against other multi-agent systems are valuable, but they leave open whether the proposed complexity is necessary relative to these simpler alternatives. This directly affects the central motivation of overcoming the limitations of single-model analysis.

  2. The evidence-generation pipeline is incomplete, limiting assessment of both the detection mechanism and inference efficiency. Algorithm 1, line 35, and Appendix F specify that analysts consume assigned precomputed evidence, but the manuscript does not adequately explain which models generate it, what inputs they use, or how it is constructed. The first-stage output is a three-dimensional score vector, which does not by itself explain the evidence supplied to the analysts. Appendix C.4 also excludes clue extraction and precomputed evidence from token accounting. The reported token reduction therefore supports a narrower claim about online reasoning after evidence preparation; it does not establish the cost of processing a new sample through the complete system. These omissions concern a central component of the method rather than a peripheral implementation detail.

  3. Figure 1 requires more careful preparation. It is embedded as a raster image, and several variable labels visibly retain red spell-check underlines. These editing artifacts should have been removed before submission. The authors should provide a clean, legible version, preferably in vector format, with mathematical notation consistent with the main text.

Questions for authors
  1. Experiments needed to establish the contribution of collaboration: Please report (a) direct prediction using the same trained clue detector and validation-selected thresholds, and (b) a single-agent alternative using the same underlying language model, prior, evidence, and routing rule. Compare predictive performance and inference cost under the existing evaluation protocol. For samples routed to reasoning, please also report how many prior errors are corrected and how many initially correct predictions become incorrect. These results would directly establish the net contribution of the multi-agent stage.

  2. Clarification and measurement needed to establish the complete pipeline: Please specify how the precomputed evidence is generated, including the generating models and their inputs. Account for the per-sample cost of clue extraction and evidence preparation when processing previously unseen news, or explicitly restrict the efficiency claims to the online reasoning stage. If efficiency superiority over competing multi-agent methods is claimed, please report their token costs under the same accounting convention.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.