- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes TruthDebate, an adaptive multi-agent framework for multimodal deepfake news detection. The framework consists of two stages. First, a multimodal clue extraction model uses learnable clue embeddings and three reasoning branches to predict face swapping, facial emotion editing, and text fabrication. Second, a multi-agent system coordinates specialised analysts, evidence integrators, an evidence auditor, and a decision-maker. The authors introduce feedback-based analyst selection and confidence-based routing to reduce unnecessary agent calls. Experiments are conducted on the MDSM dataset across five news domains, comparing TruthDebate with zero-shot multimodal models, supervised detectors, and multi-agent baselines. The paper reports improvements in detection performance and reduced language-model token consumption, supported by component ablations and additional analyses.
Critical strengths
-
The paper addresses a practically relevant problem. Detecting semantically coherent multimodal manipulations is challenging, and the attempt to combine visual, textual, and manipulation-specific evidence is meaningful.
-
The proposed framework has a reasonably clear overall structure, combining multimodal clue extraction with specialised agents, evidence integration, and confidence-based routing. The appendix provides additional mathematical definitions, implementation settings, algorithms, and agent prompts, which help explain the intended workflow.
-
The authors provide comparisons against supervised multimodal detectors and several multi-agent configurations across five news domains, together with component ablations and token consumption measurements. The reported results show improvements over the evaluated baselines, although the limitations of these comparisons are discussed below.
Critical weaknesses
1. The novelty and technical contribution are insufficient for ICLR 2027.
Applying LLM-based multi-agent collaborative reasoning to fake news detection is no longer a novel research direction. Li et al. [1] introduced FactAgent at ECAI 2024, using an agentic workflow for fact-checking and fake news detection. Lakara et al. [2] investigated multi-agent debate for visual misinformation detection in 2024. More recently, Chen et al. [3] proposed multi-perspective rationale generation, cross-verification, and adaptive evidence weighting for multimodal fake news detection at AAAI 2026. Zhang and Sui [4] also developed an adaptive role-specific multi-agent framework for fake news detection. Wu et al. [5] further explored multi-LLM interaction and collaborative reasoning for fake news detection.
These studies have already explored several key elements of TruthDebate, including specialised reasoning roles, collaborative verification, and adaptive coordination. TruthDebate combines multimodal clue extraction, analyst selection, evidence auditing, and confidence-based routing. However, the paper does not establish what fundamental technical advance this combination provides beyond existing multi-agent detection frameworks. The individual components are largely familiar, and their integration appears incremental. I would have considered this research direction more novel in 2024. For an ICLR 2027 submission, I expect a substantially clearer technical contribution than adapting and combining established multi-agent reasoning techniques for a particular downstream task.
2. The Related Work section is seriously inadequate, raising concerns about the authors' familiarity with the field.
Multimodal fake news and deepfake detection is a well-established and highly active research area. Based on my own examination of the literature, no fewer than 50 relevant papers are published annually in JCR Q1 journals and conferences ranked CCF A/B or CORE A*/A/B. Although the authors are not expected to review every study, the coverage provided in this paper is far from sufficient to reflect the breadth and depth of existing research.
The Related Work section mainly separates multimodal detection from general multi-agent reasoning, without adequately examining the substantial body of research connecting these two directions. It also lacks a critical comparison of existing methods, their limitations, and the specific technical gaps that TruthDebate aims to address. This raises serious concerns about whether the authors have conducted a thorough literature review and are sufficiently familiar with the existing research. It also undermines the credibility of their novelty claims, as the paper does not adequately establish what genuinely distinguishes the proposed approach from previous work.
3. The paper's contribution appears better suited to a more application-oriented conference.
I recognise that ICLR welcomes research across a broad range of machine learning topics. However, I find it difficult to understand why the authors chose to submit this paper to ICLR when conferences such as WWW and WSDM are more directly aligned with fake news detection. The proposed framework mainly applies and combines established multi-agent reasoning techniques for a specific downstream task, with limited methodological novelty.
In my view, an ICLR submission should demonstrate a more substantial technical advance or introduce new ideas with broader relevance to machine learning. I do not see sufficient evidence of either in this paper.
4. The experimental evaluation relies on a single dataset and lacks comparisons on established benchmarks.
The experiments are conducted exclusively on MDSM. Although the authors evaluate five news domains, all of them come from the same synthetic multimodal deepfake dataset. This is insufficient to establish the generalisability of the proposed framework.
The paper does not evaluate TruthDebate on well-established benchmarks such as DGM⁴ [6], Weibo, Twitter, FakeNewsNet (PolitiFact and GossipCop), Fakeddit, NewsCLIPpings, and VERITE. These datasets cover different aspects of multimodal fake news detection and misinformation verification, with established evaluation protocols and competitive baselines. Although these datasets involve different tasks and annotation schemes, the authors could evaluate TruthDebate on those relevant to its detection capabilities. Without such experiments, it is difficult to assess whether TruthDebate is competitive with existing methods on established benchmarks or whether its reported improvements generalise beyond MDSM.
5. The ablation studies do not adequately establish the contribution of the multi-agent reasoning framework.
Figure 2 shows that removing the Multimodal Clue Prior decreases Macro-F1 from 65.33% to 32.40%. However, removing this prior also disables direct prediction and forces all samples through full cooperative reasoning. This changes both the available evidence and the inference procedure.
More importantly, the paper does not report the performance of the trained multimodal clue extraction model alone as a complete detector. Without this comparison, it is difficult to determine how much of TruthDebate's performance comes from the supervised clue extraction stage and how much comes from multi-agent collaboration. The authors should compare the complete method with a prior-only detector, a single-agent method using the same prior and evidence, and simpler aggregation methods under comparable computational budgets. These experiments are necessary to establish whether the additional multi-agent architecture is genuinely responsible for the reported improvements.
6. The reliability estimation and fine-grained prediction procedure need stronger justification.
The proposed feedback-guided reliability estimation is based only on incorrectly predicted manipulation dimensions. As described in Appendix C.2, reliability statistics are updated when the final system prediction is wrong. Consequently, these estimates reflect a role's performance on error-triggered cases, not necessarily its overall reliability. Nevertheless, they are used for analyst selection and evidence weighting. The paper provides no clear evidence that these estimates are well calibrated or more effective than simpler reliability measures computed on validation data.
There is also insufficient explanation of how the three binary manipulation predictions are converted into the six fine-grained classes used for Macro-F1 evaluation, particularly for combined manipulation types. The conversion and final score construction should be specified explicitly.
7. The computational efficiency claims are incomplete.
The paper emphasises reduced token consumption, reporting approximately 1,237 tokens per sample for the complete method. However, Appendix C.4 and E.2 explicitly exclude the Multimodal Clue Extraction Model and precomputed evidence from this calculation. This means that the reported token reduction does not represent the complete computational cost of TruthDebate. The method additionally requires a trained multimodal detector, evidence preparation, multiple specialised agents, and feedback-based analyst selection.
The authors should report end-to-end inference time, computational cost, and the overhead of training and preparing these components. Without these measurements, the practical efficiency advantage of the overall framework remains unclear.
8. Systematic citation formatting errors and incorrect placement of the AI use statement indicate insufficient manuscript proofreading.
The manuscript contains widespread citation formatting errors throughout the Introduction, Related Work, and other sections. For example, the Introduction includes "modalities Zhang et al. (2026)" and "representations Shao et al. (2023; 2024); Ma et al. (2024); Li et al. (2025b)", where the citations should be enclosed in appropriate parentheses. Similar problems occur repeatedly throughout the paper.
Based on my experience, the authors may have used \cite{} where \citep{} was required. These basic LaTeX errors should have been corrected before submission. Their widespread occurrence makes me question whether the authors carefully reviewed the manuscript. Furthermore, the "Use of Generative AI Tools" statement appears after the References as Appendix A, whereas the official ICLR 2027 template places the AI use statement before the bibliography. These issues are straightforward to correct, but their persistence in the submitted manuscript reflects inadequate attention to basic manuscript preparation and quality control.
References
[1] X. Li, Y. Zhang, and E. C. Malthouse, "Large language model agentic approach to fact checking and fake news detection," in Proceedings of the 27th European Conference on Artificial Intelligence (ECAI), 2024.
[2] K. Lakara, G. Channing, J. Sock, C. Rupprecht, P. Torr, J. Collomosse, and C. Schroeder de Witt, "LLM-Consensus: Multi-agent debate for visual misinformation detection," arXiv preprint arXiv:2410.20140, 2024.
[3] J. Chen, Y. Li, K. C. Ng, H. Wang, and L.-J. Zhang, "Toward multimodal fake news detection by multi-perspective rationale generation and verification," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 1, 2026, pp. 66–74.
[4] Z. Zhang and J. Sui, "Deconstruct, diagnose, and deliberate: A protocol-adaptive role-specific multi-agent framework for fake news detection," in Findings of the Association for Computational Linguistics: ACL 2026, 2026, pp. 24516–24534.
[5] L. Wu, X. Jiang, S. Sun, Y. Lei, T. Wen, Y. Wang, and M. Liu, "ZoFia: Zero-shot fake news detection with entity-guided retrieval and multi-LLM interaction," in Findings of the Association for Computational Linguistics: ACL 2026, 2026.
[6] R. Shao, T. Wu, and Z. Liu, "Detecting and grounding multi-modal media manipulation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6904–6913.
Questions for authors
-
What are the main technical advances of TruthDebate beyond existing agent-based fake news detection and multimodal verification methods, particularly FactAgent [1], LLM-Consensus [2], and the AAAI 2026 work on multi-perspective rationale generation and verification [3]?
-
Why does the Related Work section omit several directly relevant studies on multi-agent fake news detection? Could the authors provide a more comprehensive review and explain how their work fits within this established research area?
-
Could the authors evaluate TruthDebate on established independent benchmarks such as DGM⁴, Weibo, Twitter, FakeNewsNet, Fakeddit, NewsCLIPpings, or VERITE, using appropriate competitive baselines? These datasets cover different aspects of multimodal misinformation detection, and results on suitable benchmarks would help establish whether the reported improvements generalise beyond MDSM.
-
Could the authors report results for the multimodal clue extraction model alone and for a single-agent method using the same prior and evidence? How much improvement can be attributed specifically to multi-agent collaboration?
-
Why is role reliability estimated only from error-triggered samples? How does this compare with reliability estimates calculated over all validation samples?
-
How are the three binary manipulation predictions converted into the six fine-grained classes? How are combined manipulation categories and final confidence scores handled?
-
Could the authors provide end-to-end runtime and computational cost measurements, including multimodal clue extraction and evidence preparation, to support the claimed efficiency improvements?