- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies perceived optical plausibility in surveillance images, distinguishing it from perceptual image quality and emphasizing whether sufficient evidence exists to judge an optical cue. It introduces GenCQA-DB, containing approximately 55,000 authentic and generated images, with quality, plausibility, and cue-judgeability annotations on a 4,800-image subset. The proposed PACT model combines appearance entropy maps, reflection/shadow/illumination residual maps, and corresponding confidence maps through a cross-map transformer. It predicts plausibility and auxiliary image quality and edit-support outputs.
Overall, I think the paper introduces a worthwhile problem, separating perceived optical plausibility from image quality and modeling whether a cue is observable at all, and the dataset and human study could be valuable. The image-to-cue pipeline is not properly explained and cannot be reproduced. The confidence term is defined as goodness of fit, so a genuine violation lowers the confidence meant to certify it was observed, undermining the paper’s central distinction. I think more work needs to be done on the formulation and modeling side to make this work in good shape. Additionally, I believe there is an AI hallucinated reference entry with a fabricated author list, and for me this is unacceptable. However, even discounting this issue, I lean towards rejecting the paper.
Critical strengths
- The problem statement and motivation is clear and identifies a real issue in surveillance and generated imagery.
- The paper uses camera-disjoint evaluation and source-matched interventions, which I agree are good ways to test whether a model is responding to optical evidence rather than memorizing viewpoints.
Critical weaknesses
- The core method of the paper is to convert a single surveillance frame into optical states and confidence maps. However it is not clear to me how reliable the obtained optical residuals are seeing they need to extract a lot of scene intrinsics such as object-shadow correspondence and normal maps. The authors need to explain further what happens if the one or more of the scene intrinsics is ambiguous or challenging to extract.
- With a free per-region albedo fit by plain least squares, a glowing region can be explained by a larger albedo instead of a residual; the paper does not say how this is prevented.
- What happens when both the evidence is poor and there is a genuine violation of the image model? It seems to me the confidence map cannot distinguish “unmeasurable” from “measured and wrong". The reliability term in Eq. 5 is defined as the goodness of fit of the optical model, and a genuine violation also worsens the fit, since the violating element is part of the data being fit.
- In the comparison to baselines, PACT receives auxiliary image-quality and edit-mask supervision, whereas the baseline training description specifies the same loss only on plausibility scores. Without matching these additional training signals, the performance advantage cannot be attributed solely to the proposed evidence representation.
- The reference contains possible AI hallucinated references: One bibliography entry (line 619 page 12) for HEIE contains the literal text: “Author list and venue to be verified.” In my view this strongly indicates that and this reference is AI-hallucinated and not properly vetted by the authors. This also contradicts with the authors AI disclosure that states that they only use AI for "proofreading and language editing". The SAC does not agree with me on this. however i still strongly suspect AI hallucination here. I have never seen this kind of reference error in any paper I have reviewed before.
Questions for authors
- Please provide reproducible details on the optical state pipeline.
- For the illumination model, what prevents the fitted region-specific albedos from absorbing a brightness anomaly instead of producing a residual? Are there constraints on albedo or an independent reflectance estimate?
- Which generators constitute GenCQA-DB, which is held out, is the joint-holdout result from one pipeline or leave-one-out, and can you report a generator-shift comparison with fixed test composition plus a plausibility-preserving edit control?
- Please provide more details on human studies such as participant count, ratings per image, and raters’ instructions for scoring an undecidable cue.
- Please explain to me the issue of the hallucinated reference, and how it contradicts with your own AI disclosure.