- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies whether video language models can recover personal information briefly exposed in public videos. It introduces VIDEO-PIE, with 170 annotated videos and a precision audit of 5,346 additional videos, and compares direct prompting with multi-stage extraction. It also proposes AutoPurify, which uses localized exposures to guide adversarial perturbations. Evaluation covers three model families and two platforms, showing substantial extraction gains from task decomposition and reduced extraction after protection. Visual distortion is evaluated on 100 frames.
Critical strengths
-
A useful empirical privacy study. Timestamped annotations and human verification provide concrete evidence of recoverable information across models and platforms. The distinction between the annotated benchmark and the larger precision audit makes the scope of the evidence clear.
-
Substantial gains from multi-stage extraction. For 20–40 minute videos, Qwen recall increases from 0.13 to 0.63 under matched video access. This is a consequential finding for assessing exposure in longer videos.
Critical weaknesses
-
The protection results need reconciliation. Table 5 implies different original baselines for the paired Frame and ROI evaluations. For Qwen Direct, the reported recall and changes imply original recalls of 0.13 and 0.18, respectively. The claimed 55–72% recall reduction also needs an explicit aggregation rule: the Qwen Multi/Frame entry implies approximately 15%. These discrepancies affect a central quantitative claim.
-
Essential protection settings are unspecified. Appendix D provides pseudocode but omits the exact backbone ensemble, representation distance, perturbation budget, step size, and iteration count. These choices directly affect protection efficacy and visual distortion, making Tables 5–6 difficult to reproduce from the paper.
Questions for authors
-
Can you provide the original baselines underlying Table 5, reconcile the Frame/ROI discrepancies, and explain how the 55–72% reduction is calculated?
-
What exact backbones, distance function, perturbation budget, step size, and iteration count were used? Please also specify the evaluation videos and aggregation procedure for Tables 5–6.