acceptodds
Withdrawn from ICLR 2027

Beyond Pixels: Bridging the Cognitive Gap in AI-Generated Video Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent advances in video generation have produced highly realistic and temporally coherent videos that are increasingly difficult to distinguish from real content. While beneficial for many applications, such progress also raises concerns such as misinformation. Existing detection methods mainly rely on pixel level artifacts and shallow spatiotemporal inconsistencies, which often fail as generation quality improves. We argue that detecting AI-generated videos of high quality requires deeper cognitive understanding beyond low level signals. Thus we propose a framework that defines five cognitive scenarios, each containing different levels of cognitive information, ranging from perception and physical plausibility to semantic, logical, and social reasoning. Through a study on how human and current MLLM detect AI-generated videos, we show that detection performance differences across various cognitive scenarios, which are related to the quality of the AI-generated video and the video understanding abilities of humans and models. Building on these findings and our proposed cognitive framework, we introduce an Iterative Cognitive Layered Instruction Evolution framework that enables multimodal models to evolve detection instructions under structured cognitive guidance. Experiments demonstrate improved detection performance and strong generalization across models and unseen AI-generated videos.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI-generated video detection, Human visual cognition, Video understanding

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jWKrOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This work studies AI-generated video detection through a human-inspired cognitive lens. It defines five cognitive scenarios (Zero-Knowledge, Perception & Senses, Physical Plausibility, Semantics & Narrative, Reasoning & Logic, Social & Cultural), measures how humans and MLLMs detect fake videos under each, and proposes an Iterative Cognitive Layered Instruction Evolution framework: a training-free loop where a multimodal model iteratively refines scenario-specific detection prompts via self-reflection. Improvements over base prompts and competitive AUC against supervised detectors are reported on CogDetect and GenVideo-val.

Critical strengths

The human study surfaces a clean cognitive-interference effect (low-level perceptual guidance hurts detection on low-quality videos, while higher-level reasoning recovers it). However, this observation is never leveraged to yield a technically novel detection mechanism—it is cashed out only as a prompt-tuning loop.

Critical weaknesses
  1. No method flowchart and no interpretable formula. The paper omits any pipeline diagram of the Iterative Cognitive Layered Instruction Evolution loop (artifact extraction → diagnosis → refinement → cross-scenario merging). Combined with the absence of any analytical treatment—e.g., error decomposition, information-gain bound, or attribution over cognitive layers—the reported SR/AUC deltas cannot be causally diagnosed. Eqs. (1)–(2) are pure bookkeeping.

  2. Innovation is a direct assemblage of known modules. The five scenarios are a straightforward repackaging of standard VLM capability dimensions (perception, physics, semantics, reasoning, social norms). The three-step evolution operator—identify artifacts, diagnose the instruction, sharpen criteria—is vanilla self-reflection / iterative prompt self-improvement already ubiquitous in LLM/VLM agent work. There is no new architectural primitive, no learnable component, and no ablation isolating cognitive-layer decomposition from flat iterative prompt tuning.

  3. The paper reads as a prompt-engineering user study, not a transferable method paper. The human cognitive-interference finding is translated, in the method section, into a training-free self-evolution prompt loop with zero novel detection mechanism. Consequently, the work offers negligible takeaway for video forensics or multimodal reasoning and its technical contribution cannot clear the ICLR bar for either area.

Questions for authors
  1. Could you provide a proper pipeline flowchart of the instruction-evolution loop? Without it, the framework is not reproducible.

  2. Can you supply any interpretable analysis (error decomposition, layer-wise attribution, or a bound on when evolution helps vs. hurts) instead of only end-task SR/AUC?

  3. What ablation compares layered cognitive instruction evolution against simply running iterative self-refinement on one flat detection prompt? If the layered structure adds nothing over the flat baseline, the cognitive hierarchy is cosmetic.

3 comments
AuthorsOpenReview ↗

To Reviewer jWKr (Part1)

We really appreciate your time and insightful comments. The response to your concerns are summarized as follows.


W1 & Q1: No method flowchart and no interpretable analysis; Eqs. (1)–(2) are bookkeeping.

A1: (1) We will add a pipeline flowchart covering the full loop: seed video selection, per-scenario artifact extraction, diagnosis, constrained refinement, iteration to N, cross-scenario merge, and the single verification pass. We agree that the current presentation describes the loop in prose only, and that this is a gap in reproducibility rather than a property of the method.

(2) On interpretable analysis, we would point out that the paper already contains the layer-wise attribution the reviewer asks for, but presents it in an appendix as a set of scenario-mixing results rather than as an attribution table. Table 16(b) reports the evolved instruction under every cumulative configuration of the hierarchy. Read as attribution, it gives the marginal contribution of each cognitive layer:

ConfigurationEvolved SR (%)ΔSR vs. baseMarginal gain of the added layer
Perception only54.11+1.40+1.40
Physical only53.80+1.09+1.09
Semantics only54.07+1.36+1.36
Reasoning only56.23+3.52+3.52
Social & Cultural only59.75+7.04+7.04
Perception + Physical55.29+2.58+1.18 over Perception
+ Semantics61.52+8.81+6.23
+ Reasoning71.65+18.94+10.13
+ Social & Cultural77.09+24.39+5.44

The marginal gains are not uniform: they are small at the perceptual layers and largest at the reasoning layer, where the criteria the model cannot supply on its own are concentrated. This is a layer-wise attribution computed from the task metric, and we will move it into the main text as such.

(3) On error decomposition, Appendices B.3 and B.4 report FNR and FPR for every scenario and every generator (Figs. 8–9). They show a systematic trade: as the guidance moves upward, missed detections fall (FNR decreases for most generators) while false alarms rise, because higher-level criteria make the model more willing to reject a video for a narrative or causal inconsistency that a real video may also exhibit. We will present this trade explicitly, since it is the mechanism by which the layers act and it is currently visible only to a reader who compares two appendix figures.

(4) We do not claim a bound on when evolution helps and when it hurts. The conditions we can state are descriptive: the gain is larger at higher layers, larger for models with stronger understanding, and larger on higher-quality generators. A predictive bound would require modelling the evolution operator, which we have not done.


W2: Innovation is an assemblage of known modules; there is no ablation isolating the cognitive-layer decomposition from flat iterative prompt tuning.

A2: (1) We accept the description of the components: the five scenarios are derived from the visual-cognition literature, and the refinement operator is an instance of iterative self-improvement. Our claim is not that either component is new. It is that the decomposition into cognitive layers changes what the refinement can do, and that this is measurable.

(2) Two existing results support the claim that the decomposition is doing work. First, the layers are not interchangeable: Table 16(b) shows that evolving one layer produces between +1.09 and +7.04 depending on which layer, and that the layers are complementary rather than redundant, since the cumulative gain exceeds the sum of the single-layer gains (+24.39 against a sum of +14.41). Second, the structure is not a presentation choice: Table 16(a) applies the same criteria in a single flat prompt without evolution and the result is worse than no guidance at all. If the decomposition were cosmetic, supplying all five scenarios at once would at least not hurt.

(3) What we do not have is the direct comparison the reviewer asks for, namely layer-wise evolution against flat iterative refinement. We will not claim that the comparison has been made.


AuthorsOpenReview ↗

To Reviewer jWKr (Part2)

W3 & Q3: The work reads as a prompt-engineering user study rather than a transferable method paper.

A3: (1) We would like to separate two questions here: whether the method is transferable, and whether the contribution is of the kind ICLR should publish.

On transferability, the framework is applied without modification to ten models across three architectures and three benchmarks, with the gains reported in Fig. 4 and Table 15 (+24.39, +20.29, +26.39 points on GenVideo-val, GenVidBench and AIGVDBench). The evolved instruction is produced once, from seven videos, and is reused across all of them.

(2) On the nature of the contribution, the deliverables are three, and only the third is a prompt.

  • First, CogDetect, a 2,411-video dataset spanning six generative models and a real-video set at controlled quality tiers, with a 341-video human-evaluation subset.
  • Second, a human study of 24 participants and 1,584 judgments with per-scenario Wilson intervals, establishes the interference effect: low-level perceptual guidance lowers human detection on low-quality videos while higher-level guidance raises it, a pattern that is not visible in any single aggregate number.
  • Third, the finding that this interaction also governs model behaviour, with the models that benefit most from higher layers being those with stronger understanding (Finding 2), which is what makes a per-layer evolution procedure sensible in the first place.

(3) We would also note what the framework does not claim to be. It is not a new detector architecture, and it does not internalise the cognitive structure into the model; it operates at inference time and is bounded by the model's own understanding, as stated in the Limitations. We regard this as the appropriate scope for a diagnosis-then-instruction approach, but we understand that a reviewer looking for an architectural contribution will find none, and we will not overstate the framing in the revision.

AuthorsOpenReview ↗

This Is a Kind Reminder for Reviewer jWKr

Dear Reviewer jWKr,

Thank you again for taking the time to provide valuable feedback on our paper. We kindly ask if you’ve had a chance to review our responses.

We hope we have adequately addressed your concerns and clarified any ambiguities. Should you have any further questions or suggestions, we would be happy to respond promptly.

Additionally, if our responses and the newly added experiment have satisfactorily resolved your concerns, we would greatly appreciate it if you could reflect this in your review score.

Thank you once again for your time and consideration.

Best regards, The Authors

Reviewer_dyhUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies AI-generated video detection using multimodal large language models (MLLMs) without additional model training. The authors first introduce five cognitive scenarios, ranging from low-level visual perception to high-level reasoning. They then compare human and MLLM detection performance under different forms of cognitive guidance. The results show that cognitive guidance can affect detection performance, and that different models respond differently to these prompts.

Building on these findings, the authors propose an iterative instruction evolution method. They use AI-generated videos to update the detection instructions under the five cognitive scenarios. The evolved instructions are then combined into a single prompt for video detection.

The proposed method improves detection performance over the base prompt on several datasets. The authors also evaluate its effectiveness across different MLLMs and video generation sources.

Critical strengths
  1. Interesting analysis of cognitive guidance for AI-generated video detection. The authors evaluate several MLLMs under five cognitive scenarios and analyze how cognitive guidance affects their detection performance.

  2. The method is simple and shows promising improvements without fine-tuning. Also, the authors evaluate the evolved prompt across different MLLMs. This suggests that the learned instructions may be useful beyond a single model.

Critical weaknesses
  1. The necessity of iterative instruction evolution is not fully demonstrated.

My main concern is whether the iterative evolution process is truly necessary. Looking at the final evolved prompt in Figure 18, most instructions are descriptions of possible artifacts, such as unnatural hand movements, inconsistent lighting, and object transformations. Some of these descriptions are also quite similar. A simpler approach could be to ask an MLLM to analyze the same seven videos under the five cognitive scenarios, extract all possible artifacts, and directly combine them into one prompt. This would avoid updating the instruction after each video. I noticed that Table 16 already compares directly combining the cognitive scenarios with the evolved instructions. However, I am not sure whether this comparison is fundamentally different from the simpler baseline described above.

  1. Limited analysis and reproducibility of the instruction evolution process.

In Section 5.1, the authors describe three steps for instruction evolution: artifact identification, missing-criterion discovery, and instruction refinement. However, some implementation details still appear to be missing. For example, how are the missing criteria identified in practice, and what prompts are used for instruction updates? Another concern is that the paper only presents the final evolved prompt in Figure 18. It would be helpful to show how the prompt changes over the seven iterations, at least for one cognitive scenario.

The authors claim that the evolution process can transform instance-specific artifacts into more general detection rules. Without showing the intermediate prompts, however, it is difficult to understand how this transformation occurs.

I would also like to see the detection performance after each iteration. This would help determine whether the iterative process provides genuine benefits, rather than merely adding more artifact descriptions to the prompt.

Questions for authors

Q1. Comparison with a simpler method: Could the authors compare the proposed method with an artifact-extraction baseline? For example, the same seven videos and five cognitive scenarios could be used to extract artifacts from each video, which would then be combined into one final prompt without iterative updates. This would help demonstrate whether the iterative process is truly necessary.

Q2. Intermediate prompts during evolution: Could the authors provide the prompt after each iteration? For example, in the Physical Plausibility scenario, it would be helpful to see how the initial prompt changes from iteration 1 to iteration 7.

Q3. Implementation of the evolution operators: Could the authors explain how Φ\Phi and Es\mathcal{E}_s are implemented? Are they performed using the same MLLM? What are the actual instructions used for artifact extraction and prompt updating?

Q4. Role of the cognitive scenarios: Have the authors tried iterative prompt refinement without using the five cognitive scenarios? For example, the model could directly analyze possible fake artifacts and update a general detection prompt.

Q5. Effect of video selection: How were the seven videos selected? Could the authors provide their sources and generation models? Because the instructions are updated one video at a time, I also wonder whether changing the video order would affect the final results. It would be useful to test several different video orders or selections and report the average performance and variation.

Q6. Prompt complexity during evolution: In Section 4.4, the authors show that overly complex prompts can reduce detection performance. Does the proposed evolution method include any mechanism to prevent the prompt from becoming too complex after several iterations?

3 comments
AuthorsOpenReview ↗

To Reviewer dyhU (Part1)

We really appreciate your time and insightful comments. The response to your concerns are summarized as follows.


W1 & Q1: The necessity of iterative evolution is not demonstrated; a single-pass artifact extraction followed by combination might suffice.

A1: (1) We agree this is the right question to ask of the method, and we will state our answer as a claim about what the iterations do rather than as a claim that one pass cannot work.

The seven iterations do not add artifacts to a growing list; they convert instance-specific observations into general decision rules. The prompt in Fig. 18 is the evidence: it contains criteria such as "identity morphing", "camera movement that is too smooth or floats unnaturally" and "lighting that produces no consistent shadows", none of which is a description of a particular video among the seven. A single pass over one video yields observations about that video; the criterion has to be abstracted before it transfers, and it is the repetition across videos that forces the abstraction.

(2) The second reason concerns what a list of artifacts can support. Table 16(a) supplies the same criteria, complete, in a single prompt, and every such configuration is worse than the Zero-Knowledge baseline. The criteria are not useful merely by being present; they have to be ordered and bound so that the model knows which to apply and when. That ordering is what the per-scenario evolution produces.

(3) We acknowledge that neither of these arguments is a controlled comparison against one-pass extraction, and we do not claim that such a comparison would necessarily favour iteration. We state it as the limitation it is.


W2 & Q2: Intermediate prompts during evolution are missing; how does instance-specific become general?

A2: We will add the instruction after each of the seven iterations for the Physical Plausibility scenario to the appendix, together with the artifact extracted at that iteration and the diagnosis that led to the update. This makes the transformation visible step by step rather than only in its final form.

The trajectory we expect the reader to see is the following, and it is what Fig.18 reflects in its final state. Iterations 1–2 produce observations tied to particular videos, for example that a hand appears to pass through an object in one clip. Iteration 3 restates this as a criterion about object interaction under occlusion. By iteration 5 the criterion has lost its reference to hands and objects and has become a statement about whether contact and support relations are preserved over time, which is what allows it to fire on a video of a different kind. This is also why the evolved criteria are grouped under the five scenarios rather than left as a flat list: the scenario supplies the level of description at which the abstraction should be expressed.


W3 & Q3: How are the evolution operators implemented, which model performs them, and what instructions are used?

A3: The evolution is performed by the same multimodal model that is later evaluated, with its parameters unchanged; no gradient is computed and no separate criterion is used. Each application of the operator consists of three constrained steps, all of which are prompts sent to that model.

  • ① Artifact identification. The model is shown video v_k together with the current instruction I_s^(k) and asked to describe, in the terms of scenarios, what in the video is characteristic of a generated video, and to quote the specific evidence in the video. The output is textual.

  • ② Diagnosis. The model is shown the artifacts from ① together with the current instruction and asked which of the artifacts the instruction failed to anticipate, i.e., which criteria are missing or too imprecise to have caught them.

  • ③ Refinement. The model is asked to rewrite the instruction so that the missing criteria are covered, under the constraint that existing criteria are preserved and that no criterion refers to a specific video, object or scene. This constraint is what enforces the move from instance-specific to general.

The merge across scenarios is a final step in which the five evolved instructions are combined into one prompt, with duplicated criteria removed and the remainder grouped under the scenario headings shown in Fig.18.


AuthorsOpenReview ↗

To Reviewer dyhU (Part2)

Q4: Has iterative refinement been tried without the five cognitive scenarios?

A4: We have not run refinement on a single flat prompt, and we will not present the hierarchy as established in isolation. What the paper does provide is the closest available comparison, in Table 16(b), where the evolution is applied to one scenario at a time and the cumulative configurations are built up. If the scenario decomposition were inert, the gain would not depend on how many layers are evolved; in fact it grows monotonically with the number of integrated layers, from +1.40 for a single layer to +24.39 for all five. That pattern is consistent with each layer contributing criteria the others cannot supply, but we agree that it is not the same as showing the flat baseline underperforms, and we will say so.


W5: How were the seven videos selected, and would a different order change the result?

A5: The seven videos are randomly selected from the CogDetect dataset and span the range of quality described in Sec. 4.1, so that evolution sees both obvious and subtle artifacts. We will report their sources and generative models in the appendix, which is currently missing and which we agree is needed for reproducibility.

In order, we have not varied it, and we will not claim invariance. What we can say is that the evolved instruction is not tied to those seven videos: it is evaluated on three benchmarks of which two are never used during evolution, and it transfers to them with gains of +20.29 points on GenVidBench and +26.39 on AIGVDBench (Table 15), as well as across ten models in Fig. 4. An instruction that had memorised the order-specific details of seven videos would not produce that transfer. The size of the seed set also cuts both ways here: seven videos is small enough that the final prompt cannot encode much about them, which is consistent with the general criteria in Fig. 18 and with the cross-dataset results, but it also means that a different draw would produce a different instruction, and we state this as a limitation.


Q6: Does the evolution include a mechanism to prevent the prompt from becoming too complex?

A6: The prompt does not grow without bound, for three reasons, although there is no explicit length cap.

  • ① The stopping rule bounds the number of updates. Each scenario is updated N = 7 times, once per video, so the instruction accumulates at most seven rounds of refinement, and the process ends because the seed videos are exhausted rather than because a threshold is met.

  • ② The refinement step is constrained to sharpen existing criteria rather than to append new ones (Sec. 5.1), so later iterations tend to merge and generalise rather than to add.

  • ③ The merge step removes redundancy across the five scenarios before the instructions are combined, which is why the final prompt of Fig. 18 contains 27 criteria rather than the sum of five independently grown lists.

That said, the reviewer's concern is well taken in one respect: the model's sensitivity to prompt complexity, documented in Sec. 4.4 and Fig. 3(c), means that the evolved prompt operates close to the point where additional detail begins to hurt. Complexity control is therefore a genuine design constraint, and we will say so in the discussion rather than implying that the merge step guarantees it.

AuthorsOpenReview ↗

This Is a Kind Reminder for Reviewer dyhU

Dear Reviewer dyhU,

Thank you again for taking the time to provide valuable feedback on our paper. We kindly ask if you’ve had a chance to review our responses.

We hope we have adequately addressed your concerns and clarified any ambiguities. Should you have any further questions or suggestions, we would be happy to respond promptly.

Additionally, if our responses and the newly added experiment have satisfactorily resolved your concerns, we would greatly appreciate it if you could reflect this in your review score.

Thank you once again for your time and consideration.

Best regards, The Authors

Reviewer_JdYFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces five cognitive guidance scenarios for AI-generated video detection and studies their effects on human and multimodal-model judgments. It further proposes an iterative instruction-evolution method using seven generated videos, reporting improved detection performance across multiple models and datasets without updating model parameters.

Critical strengths

The human–MLLM comparison and evaluations across multiple models and datasets provide useful empirical evidence about how detection instructions affect judgments of AI-generated videos without updating model parameters.

Critical weaknesses
  1. Limited methodological novelty and insufficient attribution to the cognitive hierarchy. The method primarily refines detection instructions through model-generated observations from seven videos. Comparisons with ordinary prompt refinement and length-matched detection checklists are missing, leaving unclear whether the gains require the proposed five-level hierarchy or simply more detailed instructions. The independent value of the central design is therefore not established.

  2. Several central conclusions are insufficiently supported by the reported evidence. Human accuracy does not improve monotonically across the proposed hierarchy, and significance tests against 50% chance do not establish differences between guidance conditions. The claim of consistent cross-model improvement also conflicts with Figure 4, where Qwen3-32B declines from 60.0% to 56.7%. Furthermore, the paper reports AUC without clearly explaining how continuous detection scores are obtained from the displayed binary-output protocol. These issues weaken confidence in the interpretation and reproducibility of the results.

Questions for authors
  1. How does the five-level hierarchy compare with ordinary prompt refinement and a length-matched detection checklist?

  2. Do differences between cognitive-guidance conditions remain significant after accounting for repeated judgments by the same participants and shared videos?

  3. How are continuous detection scores obtained for AUC computation, and how are classification thresholds selected?

3 comments
AuthorsOpenReview ↗

To Reviewer JdYF (Part1)

We really appreciate your time and insightful comments. The response to your concerns are summarized as follows.

W1 & Q1: Limited methodological novelty; the gains may come from more detailed instructions rather than the five-level hierarchy, and comparisons with ordinary prompt refinement and a length-matched checklist are missing.

A1: (1) We agree that the decisive question is whether the hierarchy contributes anything beyond longer instructions, and we would point to two analyses already in the paper that test exactly this.

  • ① The first is the scenario-complexity study in Sec. 4.4 (Fig. 3(c)), which varies the amount of descriptive detail while holding the task fixed. If length or detail were the driver, the Complex setting, which adds roles, styles and highly detailed cognitive descriptions, should perform best. It does not: averaged over the nine models, Basic outperforms Complex in every scenario (e.g., Perception 55.54 vs. 55.09, Physical 55.57 vs. 52.14, Reasoning 57.77 vs. 55.00, Culture 57.38 vs. 55.05). More instruction length does not help; at some point it distracts.

  • ② The second is Appendix G.2, Table 16(a), which is the closest existing analogue to a length-matched checklist: all five scenarios are supplied at once, in full, without evolution. Every mixed configuration is worse than the Zero-Knowledge baseline (ΔSR −1.01, −0.10, −0.70, −0.15), whereas each scenario used alone is better (+1.22, +1.27, +2.83, +1.11, +2.02). A checklist that merely lists the same criteria therefore degrades performance; the structure under which the criteria are applied is what matters.

(2) What distinguishes our setting from ordinary prompt refinement is that the refinement is performed per cognitive layer and then merged, rather than on one flat prompt. Table 16(b) shows why this decomposition is consequential: the marginal benefit of evolution grows with the level of the layer being evolved, from +1.40 for Perception alone to +7.04 for Social and Cultural alone, and the cumulative gain grows as layers are integrated (+2.58 for two layers, +8.81 for three, +18.94 for four, +24.39 for all five). A single flat refinement cannot allocate its update across layers in this way, because it has no layer at which to diagnose that a criterion is missing.

(3) We also want to be explicit about what these analyses do not establish. None of them is a controlled comparison against iterative refinement applied to one flat detection prompt. We have instead argued the point from the two ablations above, and we state this limitation plainly in the revision rather than treating the hierarchy as established by construction.


W2 & Q2: Human accuracy does not improve monotonically across the hierarchy, and tests against 50% chance do not establish differences between guidance conditions.

A2: (1) On monotonicity, we would clarify that the paper does not claim a monotone increase for humans. Human SR is 77.10 (None), 68.10 (Perception), 76.00 (Physical), 75.70 (Semantics), 82.60 (Reasoning) and 82.30 (Culture) in Table 10. The dip at the Perception level is not a deviation from our account; it is the cognitive interference effect that constitutes our first finding, namely that low-level perceptual guidance causes observers to over-attend to surface quality and to reject genuine videos that look degraded. The monotone claim in the paper concerns models with strong understanding capability (Finding 2), and the human claim is the heterogeneous one (Finding 1). We will make this separation explicit in the statement of the findings, since the current wording invites the reading the reviewer describes.

(2) On significance, the reviewer is right that Table 10 tests each condition against chance rather than against the other conditions, and that the overlap of the Wilson intervals is consistent with the between-condition differences being small. We should also note that a naive between-condition test would be anti-conservative here, because the same 24 participants judge multiple videos and the same videos appear across several conditions, so the judgments are not independent.

(3) The reason this does not undermine our conclusions is that our findings are interaction claims rather than main-effect claims over the hierarchy. Finding 1 states that the effect of a guidance level depends on the quality of the generated video, and Finding 2 that it depends on the model's understanding capability. Both are supported by the per-source breakdown in Tables 4–9, where the same scenario moves SR in opposite directions across the six generators: for example, Perception reduces human SR on the low-quality StepVideo subset (91.7 to 66.7) and on HunyuanVideo (75.0 to 64.6), while on the high-quality Veo3.1 subset it raises SR from 50.0 to 64.6. A single between-condition test averaged over all sources would average this interaction away, which is precisely why we report the breakdown.


AuthorsOpenReview ↗

To Reviewer JdYF (Part2)

W3: The claim of consistent cross-model improvement conflicts with Fig. 4, where Qwen3-32B declines from 60.0% to 56.7%.

A3: The reviewer is correct, and we will revise the wording. Fig. 4 reports nine improvements and one decline across ten models, with the average moving from 52.70% to 77.09% (+24.39 points); the decline is on Qwen3-VL-32B (60.0 to 56.7, −3.3). We will replace "consistent improvement across models" with "improvement on nine of ten models, with an average gain of 24.39 points", and discuss the exception. Two observations about it.

  • First, Qwen3-VL-32B is the model whose base SR is already the highest of the large models in Fig. 4, so it has the least headroom and the smallest sample of videos leaves its estimate the noisiest.

  • Second, the same model is the one whose single-scenario behaviour in Table 11 is the least differentiated across scenarios (54.4 to 59.3), which is consistent with the evolved prompt adding criteria that this model already applies rather than filling a gap. We will state this as an observation, not as an explanation we have verified.


W4: AUC is reported without explaining how continuous detection scores are obtained from a binary-output protocol.

A4: We will state this explicitly in the revision. The detector is not restricted to emitting a single token: the reported binary decision is a thresholded version of a continuous score, namely the model's probability mass assigned to the "AI-Generated" option at the decision position, and the threshold is 0.5, which is the mapping that produces the decisions in the main tables. This is what Appendix B.1 means by computing all metrics "before rounding": SR, Recall, FNR and FPR are computed on the thresholded decision, while AUC is computed on the un-thresholded score, so that the ranking is not destroyed by the threshold. The same positive/negative convention (real = positive) is applied to every model, and the prior detectors compared in Table 1 output such scores natively. We will add the score definition and the threshold to Sec. 4.4 and a worked example to Appendix B.1.

AuthorsOpenReview ↗

This Is a Kind Reminder for Reviewer JdYF

Dear Reviewer JdYF,

Thank you again for taking the time to provide valuable feedback on our paper. We kindly ask if you’ve had a chance to review our responses.

We hope we have adequately addressed your concerns and clarified any ambiguities. Should you have any further questions or suggestions, we would be happy to respond promptly.

Additionally, if our responses and the newly added experiment have satisfactorily resolved your concerns, we would greatly appreciate it if you could reflect this in your review score.

Thank you once again for your time and consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.