- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper builds an entropy-based graph for no-reference video quality assessment. A small readout predicts quality, while spatial and motion-compensated temporal edges provide localization maps. It also introduces D-CQA, a CCTV dataset with local quality labels, masks, and onset annotations, and reports quality, localization, transfer, and multimodal-context results.
Critical strengths
-
Interpretable localization signals. The metric produces interpretable localization maps: spatial responses reflect deviations from expected image statistics, while temporal responses reflect changes after motion compensation.
-
D-CQA annotation design. D-CQA combines local quality ratings with degradation masks and onset annotations. This annotation design supports separate evaluation of perceived quality, spatial extent, and temporal onset.
Critical weaknesses
-
The guarantees and localization ablations are inconsistent with the method. The proof of Proposition 4 treats CED in Eq. (5) as an average of the conditional entropies in Eq. (4), overlooking the absolute value. Taking the absolute value turns sampling errors around zero into positive contributions, so the bias can decrease more slowly than claimed. The refinement proof in Proposition 5 assumes that smaller regions have histograms close to that of their parent. As regions shrink, they can become nearly homogeneous while the parent still contains a mixture of different textures or intensities, so their histograms can remain far apart. There is also an experimental inconsistency: the localization map in §3.6 depends only on the graph, so changing readout depth alone cannot change its AUC on the same graphs and test data. Tables 2 and 4 nevertheless report such changes.
-
The evaluation does not establish an advance over existing local VQA. Patch-VQ already provides human ratings of local video regions and predicts space-time quality maps, contradicting the broad firstness claim. Its omission leaves the comparison incomplete. EFG also receives local supervision, but comparable supervision for the baselines is not documented. The experiments therefore do not separate the value of the graph from that of additional labels. The localization metric takes the strongest spatial response across the clip and discards timing, so it cannot validate the claimed ability to locate degradation onset.
Reference Zhenqiang Ying et al. “Patch-VQ: ‘Patching Up’ the Video Quality Problem.” CVPR, 2021
Questions for authors
- Which theoretical guarantees remain valid after accounting for the bias and refinement issues, and how should the localization ablations be interpreted?
- What evidence isolates the contribution of the graph from local supervision and demonstrates temporal localization?
3 comments
Thank you for re-reading the revision and for the precise follow-up. We answer the scope question first, because it decides the rest. A further revised PDF is uploaded with the changes below marked in blue.
1. Scope. EFG is intended as a method for fixed camera monitoring. The evidence in the paper supports that scope and does not support a claim of general NR-VQA, and you are right that the submission did not say so before Sec. 5.1. The abstract, the introduction, and the conclusion now state the fixed camera scope, and Sec. 5.2 describes the UGC columns of Table 1 as a robustness check under domain shift, not as a general NR-VQA ranking. We kept the title unchanged during the discussion and will narrow it in the final version if the chairs permit.
2. Alignment between D-CQA and EFG. We agree that the benchmark favors the method. The three points are now stated as limitations in a new paragraph of App. B and are referenced from Sec. 6.
| Your point | Our position | What bounds it |
|---|---|---|
| A static background reduces to near background subtraction | Agreed. Fixed views are the design premise of the temporal self reference | The warp absorbs camera motion (Fig. 6, from 0.269 to at most 0.038), but content motion still triggers (Fig. 8), which we measure as 7.1% false positives on clean clips (Table 4) |
| Local masks with sharp borders favor CED | Partly agreed. The masks are feathered, so there is no step edge, but the transition band is narrow and the border stays fairly sharp | The regions come from the segmenter and not from the mask, and 40% of the clips carry a global degradation with no border at all |
| No whole clip MOS is collected and is the tile mean | Agreed. This matches the area pooling of EFG even at | The external sets with whole clip MOS are the only evidence free of this alignment |
3. Transfer. You are right that training general backbones on synthetic CCTV data handicaps them. Table 1 now lists the official LSVQ trained results as reference rows, and Sec. 5.2 states plainly that EFG does not match them.
| Model | Training data | KoNViD-1k SRCC | LIVE-VQC SRCC |
|---|---|---|---|
| FAST-VQA | D-CQA | 0.784 | 0.741 |
| DOVER | D-CQA | 0.801 | 0.766 |
| EFG, | D-CQA | 0.837 | 0.789 |
| EFG | D-CQA | 0.842 | 0.795 |
| FAST-VQA (official) | LSVQ | 0.859 | 0.823 |
| DOVER (official) | LSVQ | 0.884 | 0.832 |
The transfer columns therefore answer a narrower question, namely what survives when the only training data is synthetic surveillance footage. Under that same training data EFG stays ahead of the general backbones, and we claim no more than that.
4. LSVQ protocol. This is the right experiment for a general claim, and EFG can be trained that way through Eq. (10), with the LSVQ space time patch labels as the region set. Building the graphs for the LSVQ videos does not finish before the discussion closes, so we cannot report it now and prefer not to report partial numbers. With the claim narrowed to fixed camera monitoring the paper no longer rests on it, and App. B names the LSVQ protocol as the test of whether the representation extends beyond fixed views.
Thank you for providing the detailed feedback on my questions. I increased my rating.
In the revised paper, the authors have addressed several issues regarding the proofs and the ablation study in W1 / Q1. Furthermore, several new baselines (Patch-VQ, Q-Align, SimpleVQA, FasterVQA, regional-head controls, temporal metrics) have been added to the paper.
When re-reading the revised paper in detail, there still remains one major open question in W2, i.e., how EFG relates to general NR-VQA? First, the paper presents EFG as a general NR-VQA method, without even mentioning CCTV until §5.1, while D-CQA uses fixed surveillance views with synthetic corruptions that are naturally aligned with the inductive biases of EFG. For example, the static background in D-CQA reduces the temporal self-reference w_t to near background subtraction (whereas content motion triggers w_t on clean walkers in Fig. 8), the pasted local masks create sharp borders that are specifically designed for CED (w_s), and no whole-clip MOS is collected—the global target y is simply the 4×4 tile mean, which matches the area pooling of EFG even for lambda_r = 0. Second, the paper investigates the question of cross-dataset transfer without standard LSVQ training (see Table 1). Training general VQA backbones on static synthetic CCTV data seems to degrade them before UGC transfer: on LIVE-VQC, D-CQA-trained DOVER (0.766) and FAST-VQA (0.741) fall well below their standard LSVQ-trained results (0.832 and 0.823, respectively).
Can the authors report EFG trained under the standard LSVQ protocol (supported by Eq. 10 and LSVQ v-patches) on LSVQ_test, LSVQ_1080p, KoNViD-1k, and LIVE-VQC, together with the official LSVQ-trained baselines, and also clarify whether EFG is intended as a general NR-VQA method or as a method for fixed-camera monitoring?