acceptodds
Withdrawn from ICLR 2027

Plausibility Is Not Quality: Observability-Aware Optical Evidence for Surveillance Image Assessment

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generative models now synthesize surveillance frames that are sharper than authentic footage yet depict scenes that could not physically occur, with reflections lacking sources and shadows lacking objects. Image quality assessment offers no protection because these frames are not degraded, and we show that perceptual quality and perceived plausibility are dissociable perceptual axes. A third property, observability, governs both, since a shadow may be cast by an object outside the frame, so an assessor must distinguish an observed violation from insufficient evidence. We formalize plausibility as the Perceived Plausibility Score (PPS), a continuous human-anchored measure whose protocol also records whether each optical cue was judged decidable, and we construct GenCQA-DB, approximately 55,000 source-grouped authentic and generated CCTV images with controlled reflection, shadow, and illumination violations. We propose PACT, a Physical and Appearance Cross-map Transformer that pairs each optical residual map with a cue confidence map encoding evidence availability and fuses them with appearance entropy maps through cross-map attention. PACT attains an SRCC of 0.874 against 0.791 for an equally supervised image baseline under camera-disjoint evaluation, retains 0.836 when cameras and generator are both held out, and follows human judgments when the evidence for a violation fades instead of mistaking a vanished residual for a consistent scene.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Perceived plausibility, Image quality assessment, Observability, Optical consistency, Physics-grounded evaluation, AI-generated image assessment, Surveillance imagery, Cross-map transformer, Subjective study

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ogqxOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies perceived optical plausibility in surveillance images, distinguishing it from perceptual image quality and emphasizing whether sufficient evidence exists to judge an optical cue. It introduces GenCQA-DB, containing approximately 55,000 authentic and generated images, with quality, plausibility, and cue-judgeability annotations on a 4,800-image subset. The proposed PACT model combines appearance entropy maps, reflection/shadow/illumination residual maps, and corresponding confidence maps through a cross-map transformer. It predicts plausibility and auxiliary image quality and edit-support outputs.

Overall, I think the paper introduces a worthwhile problem, separating perceived optical plausibility from image quality and modeling whether a cue is observable at all, and the dataset and human study could be valuable. The image-to-cue pipeline is not properly explained and cannot be reproduced. The confidence term is defined as goodness of fit, so a genuine violation lowers the confidence meant to certify it was observed, undermining the paper’s central distinction. I think more work needs to be done on the formulation and modeling side to make this work in good shape. Additionally, I believe there is an AI hallucinated reference entry with a fabricated author list, and for me this is unacceptable. However, even discounting this issue, I lean towards rejecting the paper.

Critical strengths
  • The problem statement and motivation is clear and identifies a real issue in surveillance and generated imagery.
  • The paper uses camera-disjoint evaluation and source-matched interventions, which I agree are good ways to test whether a model is responding to optical evidence rather than memorizing viewpoints.
Critical weaknesses
  • The core method of the paper is to convert a single surveillance frame into optical states and confidence maps. However it is not clear to me how reliable the obtained optical residuals are seeing they need to extract a lot of scene intrinsics such as object-shadow correspondence and normal maps. The authors need to explain further what happens if the one or more of the scene intrinsics is ambiguous or challenging to extract.
  • With a free per-region albedo fit by plain least squares, a glowing region can be explained by a larger albedo instead of a residual; the paper does not say how this is prevented.
  • What happens when both the evidence is poor and there is a genuine violation of the image model? It seems to me the confidence map cannot distinguish “unmeasurable” from “measured and wrong". The reliability term in Eq. 5 is defined as the goodness of fit of the optical model, and a genuine violation also worsens the fit, since the violating element is part of the data being fit.
  • In the comparison to baselines, PACT receives auxiliary image-quality and edit-mask supervision, whereas the baseline training description specifies the same loss only on plausibility scores. Without matching these additional training signals, the performance advantage cannot be attributed solely to the proposed evidence representation.
  • The reference contains possible AI hallucinated references: One bibliography entry (line 619 page 12) for HEIE contains the literal text: “Author list and venue to be verified.” In my view this strongly indicates that and this reference is AI-hallucinated and not properly vetted by the authors. This also contradicts with the authors AI disclosure that states that they only use AI for "proofreading and language editing". The SAC does not agree with me on this. however i still strongly suspect AI hallucination here. I have never seen this kind of reference error in any paper I have reviewed before.
Questions for authors
  • Please provide reproducible details on the optical state pipeline.
  • For the illumination model, what prevents the fitted region-specific albedos from absorbing a brightness anomaly instead of producing a residual? Are there constraints on albedo or an independent reflectance estimate?
  • Which generators constitute GenCQA-DB, which is held out, is the joint-holdout result from one pipeline or leave-one-out, and can you report a generator-shift comparison with fixed test composition plus a plausibility-preserving edit control?
  • Please provide more details on human studies such as participant count, ratings per image, and raters’ instructions for scoring an undecidable cue.
  • Please explain to me the issue of the hallucinated reference, and how it contradicts with your own AI disclosure.
Reviewer_CWn4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces GenCQA-DB and the Perceived Plausibility Score (PPS). It proposes PACT, which combines optical residuals, cue confidence, and appearance entropy maps to predict perceived optical plausibility in surveillance images.

Critical strengths
  1. The distinction between perceptual quality and optical plausibility is well motivated.
  2. The dataset includes quality, plausibility, and cue judgeability annotations.
  3. The experiments include camera and generator holdouts, ablations, and controlled interventions.
Critical weaknesses
  1. Task-aligned comparisons and forensic utility remain unverified. The evaluation lacks dedicated implausibility baselines and does not establish whether PPS adds useful information beyond existing authenticity detectors.

  2. Editing artifacts remain confounded with physical violations. Without matched physically plausible edits, the experiments cannot sufficiently distinguish optical reasoning from reliance on construction artifacts.

  3. Optical residual validity is insufficiently established. The extraction of required geometry and correspondences remains underspecified. If regional albedo is fitted without sufficient constraints, abnormal brightness may be explained by reflectance rather than detected as an illumination inconsistency.

  4. The source of the performance gains remains unclear. It is not specified whether the baselines receive the same auxiliary supervision as PACT, and comparisons with regressors using residual magnitudes and observability scores are missing.

Questions for authors

see Weaknesses

Reviewer_DCuROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors argue that a surveillance image appearing plausible does not necessarily provide sufficient physical evidence to support its authenticity; thus, image quality and plausibility should be modeled as distinct dimensions. To address this, they construct the GenCQA-DB dataset which includes real surveillance frames and manipulated images synthesized for three categories of optical violations: reflection, shadow and illumination. They formulate a Perceived Plausibility Score (PPS) to quantify the optical plausibility of visible scenes, and present a Physical and Appearance Cross-map Transformer utilizing optical evidence and observability to assess image credibility. Experiments show that PPS outperforms equally supervised image models on GenCQA-DB under camera-disjoint evaluation and under joint camera and generator holdout.

Critical strengths
  1. This paper introduces a new evaluation perspective that explicitly decouples image quality from perceived plausibility. Empirical evidence from human studies reveals that while Mean Opinion Score (MOS) and Perceived Plausibility Score (PPS) are highly correlated on authentic frames, this correlation drops significantly for generated content. This finding demonstrates that perceptual quality and plausibility can be substantially independent in synthetic imagery, offering valuable insights for both generative image assessment and visual forensics.
  2. This paper introduces the observability-aware physical evidence to assess image credibility. Instead of simply detecting whether physical laws are violated, it focuses on whether a physical law is violated and whether we have sufficient evidence to make such a judgment.
Critical weaknesses
  1. Fairness and Supervision Disparity in Baseline Comparisons. The current comparison against other baselines (such as the VLM judge family and Forensic Family) is slightly unfair. For example, the VLM judges are evaluated in a zero-shot manner, whereas PACT undergoes supervised training on PPS labels using hand-crafted physics features. Given this supervision advantage, PACT's superior performance is somewhat anticipated. To convincingly substantiate the effectiveness of PACT's dedicated design, the authors should incorporate a fine-tuned VLM baseline trained under identical supervision using the same PPS labels. Demonstrating PACT's consistent outperformance over a supervised VLM would significantly strengthen the motivation and validity of the proposed optical evidence framework.
  2. Potential Synthetic Benchmark Bias and Real-World Generalization of GenCQA-DB. GenCQA-DB relies heavily on controlled interventions for image manipulation, which provides a methodologically clean setup for causal analysis but inherently risks synthetic benchmark bias. Consequently, models may overly adapt to the specific artifacts or signatures generated by the predefined violation pipeline. In contrast, real-world generative models typically introduce complex, highly coupled physical inconsistencies, where shadows, reflections, geometry, and textures degrade simultaneously. To validate that PACT is robust for practical deployment rather than merely capturing artifacts from controlled pipelines, the authors are encouraged to evaluate the framework on unseen generative models, novel editing pipelines, and naturally occurring synthetic imagery.
Questions for authors

While the paper demonstrates strong generalization across camera-disjoint splits, generator holdouts, and source-matched interventions, all empirical evaluations remain strictly confined to the proposed GenCQA-DB dataset. How well does the proposed method transfer to a completely different, independent surveillance benchmark? Could the authors provide cross-dataset evaluation results to confirm that the model relies on universal physical plausibility principles rather than dataset-specific statistical artifacts or generator-specific signatures?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.