acceptodds
Under review as a conference paper at ICLR 2027

Camera Scene Grammars: Reference-Free Semantic Quality Assessment for Fixed-View Surveillance

Abstract

Surveillance streams are monitored through scalar quality scores, yet a single score cannot say which observations in the scene remain dependable, whether a degradation has removed a person, introduced a phantom one, or turned a parked vehicle into a moving one. We argue that a fixed camera carries a resource that isolated-image quality assessment lacks, namely its own earlier observations of the same view, and we formulate history-conditioned semantic reliability as the task of judging a degraded stream against that history without any contemporaneous clean reference. A camera scene grammar G_c stores persistent anchors, class-conditioned occupancy, track continuity, and relation context estimated from undistorted history. Comparing the stream with its grammar yields three typed evidence channels, missing expected content, unsupported observations, and inadmissible relations, together with an explicit coverage mask that marks where the history cannot support a judgment, and a grammar-conditioned student amortizes the comparison using supervision derived from clean and degraded pairs without opinion scores. We give an exact error decomposition for ensemble consensus under shared generator bias, conditional monotonicity and stability guarantees for the channels, and an identifiability limit that bounds what any history can recover. Under a camera-disjoint protocol on the 50-camera D-CQA corpus with independently annotated object-level errors, the scalar readout reaches 0.903 SRCC, per-channel attribution reaches 0.881, 0.902, and 0.858 AUROC for dropped, hallucinated, and misrelated content, and matched-history baselines, a different camera's grammar, and a spatially permuted grammar each recover only part of the gain, isolating camera-specific semantic structure as its source. We release the graph annotation layer D-CQA-SG.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
fixed-view surveillance, no-reference video quality assessment, scene graph, semantic reliability, history-conditioned assessment, object-level attribution

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_JKTJOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Camera Scene Grammars (CSG) for assessing semantic reliability in fixed-view surveillance using earlier observations from the same camera, without a contemporaneous clean reference. The method constructs persistent anchors, class-conditioned occupancy statistics, track-continuation models, and relation contexts. It produces separate evidence channels for missing content, unsupported observations, and inadmissible relations, together with coverage masks.

Critical strengths
  1. The formulation is well structured and useful. The paper moves beyond scalar quality prediction by introducing typed object-level evidence, expected-node slots, and explicit coverage.

  2. The experiments are thoughtfully designed. The evaluation includes matched-history baselines, swapped and spatially permuted grammars, alignment controls, and component ablations, rather than relying only on comparisons with history-free quality models (Table 3 and Appendix E).

Critical weaknesses
  1. The selective-decision advantage is not sufficiently supported at matched coverage. Appendix H mentions a matched-retained-coverage comparison, but the corresponding results are not shown. Table 9 reports a 6.1% unresolved fraction for CSG and none for the listed baselines, leaving unclear whether CSG’s lower false-rejection rate reflects better discrimination or differences in abstention. Please report false-rejection rates and error-detection performance at the same retained coverage.

  2. The independence of the downstream evaluation models is unclear. Section 5.1 cites Zhao et al. (2024) for the evaluation detector, while Appendix L includes RT-DETR in the ensemble and states that evaluation models come from outside the ensemble’s model families. Please specify the detector, tracker, and checkpoints used in Tables 2 and 5, and clarify their relationship to the ensemble. This would establish whether the results support the claimed applicability to model families outside the ensemble.

Questions for authors
  1. Does CSG retain its advantage over the confidence/track and history-standardized baselines at the same retained coverage? What are the false-rejection rates and error-detection performance under this comparison?

  2. What are the evaluation units and denominators for the rates in Table 9? How many events or candidate observations are included in the 36 clips, and how are they distributed across challenge categories?

  3. Which detector, tracker, and checkpoints are used in Tables 2 and 5? Are their model families represented in the ensemble?

1 comment

Thank you for the detailed response and revisions. The clarifications and additional experiments have addressed most of my concerns and strengthened the empirical support for the paper. I therefore raise my rating from Weak rejection to Weak acceptance.

Reviewer_fre3OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies semantic reliability in fixed-view surveillance streams and proposes history-conditioned semantic reliability assessment using historical observations from the same camera. Camera Scene Grammars summarize anchored objects, class-conditioned occupancy, track continuity, and relation context. Comparing current observations with history produces typed structured outputs containing consistency scores, evidence sources, and validity masks.

Critical strengths

Practical value of structured outputs. Separating missing objects, unsupported observations, and relation errors provides more specific information than an overall quality score. Expected-node slots represent some missing observations, while coverage masks explicitly mark judgments with insufficient evidence.
Method design supported by theoretical analysis. The authors organize camera-specific history into scene grammars, combining expected-node slots and evidence coverage for structured reliability assessment. The analysis characterizes how shared bias and model correlation affect ensemble error, conditional monotonicity and stability, and the identifiability limits of historical information, supporting the method and clarifying its scope.

Critical weaknesses

Historical consistency does not establish semantic correctness. A real object’s first appearance or a legitimate scene change may violate the historical grammar without indicating an erroneous observation. It remains unclear how validity masks handle a well-represented class appearing in a previously unobserved region. Incremental contribution over scene-specific anomaly detection. C-SARD [1] and BUSSARD [2] already exploit scene-specific relational regularities. The paper should clarify its substantive methodological advances beyond extending scene-normality scoring to more outputs.

[1] Lai Y C, Sonogashira M, Phueaksri I, et al. Scene-specific anomalous relationship detection using scene graph summarization[C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 2025: 3995-4003. [2] Schween M, Kruse M, Rosenhahn B. BUSSARD: Normalizing Flows for Bijective Universal Scene-Specific Anomalous Relationship Detection[J]. arXiv preprint arXiv:2603.16645, 2026.

Questions for authors

NA

Reviewer_CqW6OpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a reference-free quality assessment method for fixed surveillance cameras. Instead of only predicting a single scalar score, the authors use 30 to 40 minutes of earlier clean video from the same camera to build a scene grammar (capturing static objects, common agent locations, track duration, and object relations), and compare the current frame's scene graph against this prior to detect missing objects, false detections, and wrong relations. They also train a lightweight student network from a 5-generator teacher ensemble without using human MOS labels, and evaluate the approach on D-CQA, MOT20, and UA-DETRAC.

Critical strengths

Keeping unmatched static objects and active tracks as explicit slots in the output graph (paired with a coverage mask) is a nice design. Standard VQA and scene-graph anomaly methods have no box to attach a score to when a detector completely misses an object, and allowing the model to abstain on unseen regions noticeably reduces false alarms on rare events in Section 5.5.

The control and ablation experiments in Section 5 are convincing. Giving all baselines the same camera history in Table 1 and testing swapped or spatially shuffled grammars in Table 3 makes it clear how much of the improvement comes from simple per-camera score normalization versus the scene grammar itself.

Building the grammar and distilling the student model without human MOS annotations makes the pipeline practical to calibrate and deploy on new fixed cameras.

The evaluation is thorough, covering both human quality scores on D-CQA and downstream detection/tracking drops on MOT20 and UA-DETRAC, and the new D-CQA-SG graph annotations will be a useful benchmark.

Critical weaknesses

In Tables 1 and 3, the scalar SRCC gain over the matched-history baselines comes almost entirely from running the 5-generator ensemble, which is very slow (640 ms/frame). When using a single generator (K = 1) or the fast distilled student (CSG fast), the performance is actually slightly worse than history-standardized RA-MIQA and BUSSARD.

It is not clear from Section 5 or the appendix whether graph baselines like C-SARD and BUSSARD were evaluated with a single scene graph generator or the same 5-model ensemble used by CSG full. If they only used one generator, the comparison with CSG full is not fair in terms of compute.

Because the grammar is built from static history, the method is sensitive to normal scene changes. Things like a small camera bump, sudden lighting changes, or someone moving objects around get flagged as video corruption (Section 5.5), and even with the sliding-window update the false alarm rate is still around 10%.

Each camera needs 20 to 40 minutes of clean video before monitoring can start, and the system is bottlenecked by the fixed 12-relation vocabulary of the pretrained scene graph generators, so it cannot catch errors on object classes or relations outside that set.

Questions for authors

In Tables 1 and 2, were C-SARD and BUSSARD run with a single scene graph generator (K = 1) or with the same 5-generator ensemble used by CSG full?

For CSG fast (14 ms/frame), how are the past object tracks obtained at test time without running an external detector or tracker on previous frames?

How does the sliding-window grammar update handle slow environmental changes (like dusk or accumulating snow) where the quality score drops below the 0.8 update threshold without triggering the 90-second change-point test?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.