- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work studies AI-generated video detection through a human-inspired cognitive lens. It defines five cognitive scenarios (Zero-Knowledge, Perception & Senses, Physical Plausibility, Semantics & Narrative, Reasoning & Logic, Social & Cultural), measures how humans and MLLMs detect fake videos under each, and proposes an Iterative Cognitive Layered Instruction Evolution framework: a training-free loop where a multimodal model iteratively refines scenario-specific detection prompts via self-reflection. Improvements over base prompts and competitive AUC against supervised detectors are reported on CogDetect and GenVideo-val.
Critical strengths
The human study surfaces a clean cognitive-interference effect (low-level perceptual guidance hurts detection on low-quality videos, while higher-level reasoning recovers it). However, this observation is never leveraged to yield a technically novel detection mechanism—it is cashed out only as a prompt-tuning loop.
Critical weaknesses
-
No method flowchart and no interpretable formula. The paper omits any pipeline diagram of the Iterative Cognitive Layered Instruction Evolution loop (artifact extraction → diagnosis → refinement → cross-scenario merging). Combined with the absence of any analytical treatment—e.g., error decomposition, information-gain bound, or attribution over cognitive layers—the reported SR/AUC deltas cannot be causally diagnosed. Eqs. (1)–(2) are pure bookkeeping.
-
Innovation is a direct assemblage of known modules. The five scenarios are a straightforward repackaging of standard VLM capability dimensions (perception, physics, semantics, reasoning, social norms). The three-step evolution operator—identify artifacts, diagnose the instruction, sharpen criteria—is vanilla self-reflection / iterative prompt self-improvement already ubiquitous in LLM/VLM agent work. There is no new architectural primitive, no learnable component, and no ablation isolating cognitive-layer decomposition from flat iterative prompt tuning.
-
The paper reads as a prompt-engineering user study, not a transferable method paper. The human cognitive-interference finding is translated, in the method section, into a training-free self-evolution prompt loop with zero novel detection mechanism. Consequently, the work offers negligible takeaway for video forensics or multimodal reasoning and its technical contribution cannot clear the ICLR bar for either area.
Questions for authors
-
Could you provide a proper pipeline flowchart of the instruction-evolution loop? Without it, the framework is not reproducible.
-
Can you supply any interpretable analysis (error decomposition, layer-wise attribution, or a bound on when evolution helps vs. hurts) instead of only end-task SR/AUC?
-
What ablation compares layered cognitive instruction evolution against simply running iterative self-refinement on one flat detection prompt? If the layered structure adds nothing over the flat baseline, the cognitive hierarchy is cosmetic.
3 comments
To Reviewer jWKr (Part2)
W3 & Q3: The work reads as a prompt-engineering user study rather than a transferable method paper.
A3: (1) We would like to separate two questions here: whether the method is transferable, and whether the contribution is of the kind ICLR should publish.
On transferability, the framework is applied without modification to ten models across three architectures and three benchmarks, with the gains reported in Fig. 4 and Table 15 (+24.39, +20.29, +26.39 points on GenVideo-val, GenVidBench and AIGVDBench). The evolved instruction is produced once, from seven videos, and is reused across all of them.
(2) On the nature of the contribution, the deliverables are three, and only the third is a prompt.
- First, CogDetect, a 2,411-video dataset spanning six generative models and a real-video set at controlled quality tiers, with a 341-video human-evaluation subset.
- Second, a human study of 24 participants and 1,584 judgments with per-scenario Wilson intervals, establishes the interference effect: low-level perceptual guidance lowers human detection on low-quality videos while higher-level guidance raises it, a pattern that is not visible in any single aggregate number.
- Third, the finding that this interaction also governs model behaviour, with the models that benefit most from higher layers being those with stronger understanding (Finding 2), which is what makes a per-layer evolution procedure sensible in the first place.
(3) We would also note what the framework does not claim to be. It is not a new detector architecture, and it does not internalise the cognitive structure into the model; it operates at inference time and is bounded by the model's own understanding, as stated in the Limitations. We regard this as the appropriate scope for a diagnosis-then-instruction approach, but we understand that a reviewer looking for an architectural contribution will find none, and we will not overstate the framing in the revision.
This Is a Kind Reminder for Reviewer jWKr
Dear Reviewer jWKr,
Thank you again for taking the time to provide valuable feedback on our paper. We kindly ask if you’ve had a chance to review our responses.
We hope we have adequately addressed your concerns and clarified any ambiguities. Should you have any further questions or suggestions, we would be happy to respond promptly.
Additionally, if our responses and the newly added experiment have satisfactorily resolved your concerns, we would greatly appreciate it if you could reflect this in your review score.
Thank you once again for your time and consideration.
Best regards, The Authors
To Reviewer jWKr (Part1)
We really appreciate your time and insightful comments. The response to your concerns are summarized as follows.
W1 & Q1: No method flowchart and no interpretable analysis; Eqs. (1)–(2) are bookkeeping.
A1: (1) We will add a pipeline flowchart covering the full loop: seed video selection, per-scenario artifact extraction, diagnosis, constrained refinement, iteration to N, cross-scenario merge, and the single verification pass. We agree that the current presentation describes the loop in prose only, and that this is a gap in reproducibility rather than a property of the method.
(2) On interpretable analysis, we would point out that the paper already contains the layer-wise attribution the reviewer asks for, but presents it in an appendix as a set of scenario-mixing results rather than as an attribution table. Table 16(b) reports the evolved instruction under every cumulative configuration of the hierarchy. Read as attribution, it gives the marginal contribution of each cognitive layer:
The marginal gains are not uniform: they are small at the perceptual layers and largest at the reasoning layer, where the criteria the model cannot supply on its own are concentrated. This is a layer-wise attribution computed from the task metric, and we will move it into the main text as such.
(3) On error decomposition, Appendices B.3 and B.4 report FNR and FPR for every scenario and every generator (Figs. 8–9). They show a systematic trade: as the guidance moves upward, missed detections fall (FNR decreases for most generators) while false alarms rise, because higher-level criteria make the model more willing to reject a video for a narrative or causal inconsistency that a real video may also exhibit. We will present this trade explicitly, since it is the mechanism by which the layers act and it is currently visible only to a reader who compares two appendix figures.
(4) We do not claim a bound on when evolution helps and when it hurts. The conditions we can state are descriptive: the gain is larger at higher layers, larger for models with stronger understanding, and larger on higher-quality generators. A predictive bound would require modelling the evolution operator, which we have not done.
W2: Innovation is an assemblage of known modules; there is no ablation isolating the cognitive-layer decomposition from flat iterative prompt tuning.
A2: (1) We accept the description of the components: the five scenarios are derived from the visual-cognition literature, and the refinement operator is an instance of iterative self-improvement. Our claim is not that either component is new. It is that the decomposition into cognitive layers changes what the refinement can do, and that this is measurable.
(2) Two existing results support the claim that the decomposition is doing work. First, the layers are not interchangeable: Table 16(b) shows that evolving one layer produces between +1.09 and +7.04 depending on which layer, and that the layers are complementary rather than redundant, since the cumulative gain exceeds the sum of the single-layer gains (+24.39 against a sum of +14.41). Second, the structure is not a presentation choice: Table 16(a) applies the same criteria in a single flat prompt without evolution and the result is worse than no guidance at all. If the decomposition were cosmetic, supplying all five scenarios at once would at least not hurt.
(3) What we do not have is the direct comparison the reviewer asks for, namely layer-wise evolution against flat iterative refinement. We will not claim that the comparison has been made.