- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Mask2Safe to remove identity and sensitive attributes while keeping other relevant information in the video. Mask2Safe trains a lightweight Privacy Masker (frozen DINOv2 + small decoder) on unsafe/safe image pairs from Unsafe2Safe. It learns which regions must change by using a U-Net to predict replacement content. On video, the per-frame masks seed SAM2, which refines and tracks the region within each shot. The region is then either blacked out or regenerated with a video inpainting model. The paper also proposes SafeQA, a benchmark to tell apart if privacy is achieved by hiding content (visibility) or is achieved by changing only the sensitive parts (local utility).
Critical strengths
The paper explores replacement instead of suppression, with real-video transfer. Mask2Safe edits only localized regions so classifiers trained on its videos transfer to original videos, while Unsafe2Safe drops almost 20%. SafeQA probes the edited regions themselves, which clip-level metrics cannot do. This is a useful direction.
Critical weaknesses
The mask-learning method is not shown to be needed. The trainable U-Net (later discarded) and the L_pref and L_preserve losses are never ablated. Simpler mask sources are never compared: thresholded source and target image differences, a frozen pretrained editor, or detector + SAM2 person masks. This matters because refined masks often cover whole people (appendix figure 12). If a person segmenter performs as well, the claim that image-learned privacy knowledge transfers to video is unsupported.
Furthermore, cMAP improves only from 72.86 to 71.07 on VP-UCF101 (4× downsampling: 71.61). On SafeQA Privacy, Mask2Safe is worse than Unsafe2Safe and TeD-SPAD. SafeQA relies on a single Gemini judge with no human validation and has about 20% chance accuracy on Q1 and Q3. SafeQA is the paper’s main evidence for the benefit of replacement and this undermines the central claim.
Questions for authors
The refined masks in Figs. 11 and 12 often cover whole people. How do the learned masks compare with detector + SAM2 person masks, or with thresholded source-target latent differences used as pseudo-labels, under the same black-out and EditCtrl inpainting?
The U-Net G is discarded after training, and L_pref and L_preserve mainly train G rather than the mask. Are they needed? How do the masks change with a frozen pretrained editor (for instance, InstructPix2Pix) as G, or without L_pref, L_preserve or the stop-gradients?
Appendix A.2 prompts EditCtrl with “A video of {class name}”. Is the ground-truth label also used for test videos? If so, what is the action accuracy in Table 1 with a class-agnostic prompt?
SafeQA questions are generated and answered by Gemini models with no human check. Could you report human agreement on a random subset and results with a second (ideally open-source) judge?
7 comments
Response to Reviewer DNCq's Weakness 2 and Questions 1-2
W2. Privacy–Utility Trade-off and SafeQA Reliability (Appendices G–H)
We agree that the reduction in privacy cMAP is modest. However, cMAP measures the recoverability of predefined privacy attributes and does not fully characterize other forms of privacy leakage, such as facial identity linkability.
Thus, we provide an additional evaluation of cross-video identity leakage using ArcFace embeddings (Appendix G). Specifically, we measure facial similarity across different videos containing the same person. Ideally, anonymization should make it difficult to recognize that the same person appears in multiple videos. On VP-UCF101 and VP-HMDB51, cross-video facial similarity (lower is better) decreases from 0.769 to 0.223 and from 0.722 to 0.224, respectively, after generative anonymization. These results indicate substantially reduced identity linkability across videos.
More importantly, the benefit of our generative replacement is supported by SafeQA. Under the same privacy localization pipeline, replacing sensitive regions rather than directly masking them improves original-domain action accuracy by 9.49 and 5.03 percentage points on VP-UCF101 and VP-HMDB51, respectively. Compared with Unsafe2Safe, our method also improves original-domain action accuracy by 11.18 and 1.69 points (Table 1). These results directly support the utility of generative replacement for downstream video understanding.
To assess evaluator dependence, we extend SafeQA evaluation to three open-source VLMs: Qwen3.5-4B, Qwen3.5-9B, and Qwen3-VL-8B (Appendix H). We examine whether the conclusions regarding privacy recovery and local utility remain consistent. We report representative Gemini and Qwen3.5-4B results in the rebuttal for brevity, with additional evaluations provided in the appendix.
SafeQA results evaluated using Gemini and Qwen3.5-4B. Bold and italic indicate the best and second-best results, respectively, among anonymization methods.
| Method | Gemini Q1 ↓ | Gemini Q2 ↑ | Gemini Q3 ↑ | Qwen Q1 ↓ | Qwen Q2 ↑ | Qwen Q3 ↑ |
|---|---|---|---|---|---|---|
| VP-UCF101 | ||||||
| Raw | 97.3 | 98.1 | 94.3 | 90.0 | 96.3 | 92.8 |
| Downsample 2× | 91.2 | 97.3 | 91.4 | 84.2 | 93.9 | 90.6 |
| Downsample 4× | 78.4 | 95.6 | 87.4 | 74.9 | 80.4 | 86.2 |
| TeD-SPAD | 55.0 | 68.8 | 66.4 | 60.6 | 21.1 | 65.9 |
| Unsafe2Safe | 50.6 | 71.7 | 65.9 | 60.1 | 72.5 | 70.2 |
| Mask2Safe (Mask-based) | 62.3 | 50.6 | 74.9 | 64.8 | 25.8 | 73.0 |
| Mask2Safe (Generative) | 63.8 | 77.1 | 73.6 | 68.1 | 74.6 | 77.5 |
| VP-HMDB51 | ||||||
| Raw | 97.9 | 98.5 | 94.6 | 88.1 | 91.7 | 90.6 |
| Downsample 2× | 84.2 | 86.7 | 85.5 | 84.6 | 87.3 | 88.4 |
| Downsample 4× | 74.2 | 86.8 | 82.1 | 76.7 | 76.5 | 83.8 |
| TeD-SPAD | 58.4 | 65.5 | 68.9 | 60.8 | 24.2 | 70.0 |
| Unsafe2Safe | 49.1 | 72.0 | 64.6 | 60.6 | 69.0 | 69.2 |
| Mask2Safe (Mask-based) | 62.0 | 47.2 | 73.3 | 64.4 | 23.3 | 74.7 |
| Mask2Safe (Generative) | 71.0 | 80.3 | 77.3 | 71.1 | 77.0 | 79.9 |
We are extending SafeQA with human evaluation on the question quality and will report these results once completed.
These findings reinforce the utility benefits of generative replacement. That said, stronger privacy suppression remains an area for improvement.
Q1. Comparison Under Identical Anonymization Pipelines
As detailed in W1, our additional experiments demonstrate improved privacy localization over difference-based pseudo-labels and pretrained person segmentation.
Q2. Auxiliary U-Net, Regional Losses, and Stop-Gradient
W1 also provides ablations showing that the regional reconstruction losses are important for learning meaningful privacy masks, despite the auxiliary U-Net being discarded after training. The individual U-Net stop-gradient has limited impact, whereas removing all stop-gradients substantially degrades localization.
Response to Reviewer DNCq's Questions 3-4
Q3. Action-Conditioned Versus Class-Agnostic Editing (Appendix C.3, F.2-F.6, Table 11)
In the main evaluation, we have two different test settings for utility: (1) the accuracy on anonymized videos and (2) the accuracy on original test videos. We clarify that action-conditioned prompts are used to generate anonymized videos for training and the first setting, but not for the second one (original-domain evaluation). In the second setting, classifiers are evaluated on unmodified original videos, without editing or ground-truth action prompts at inference. Therefore, the original-domain results cannot be attributed to action-conditioned test-video generation.
To directly assess potential prompt-induced bias in same-domain evaluation, we additionally evaluate class-agnostic editing on the test videos. Instead of supplying the ground-truth action class, we use per-video edit instruction generated by Gemini. These descriptions retain observable movements, object interactions, and scene structure while modifying identifying appearance without mentioning the class name.
| Method | Test Video | Editing Prompt | VP-UCF101 ↑ | VP-HMDB51 ↑ |
|---|---|---|---|---|
| Mask2Safe (Ours) | Original | -- | 76.51 | 39.08 |
| Anonymized | Action-conditioned | 74.53 | 39.93 | |
| Anonymized | Class-agnostic (Gemini) | 72.14 | 38.82 | |
| TeD-SPAD | Anonymized | -- | 62.58 | 30.65 |
Action-recognition accuracy (Top-1, %, ↑) under different test-time editing settings. Full results are reported in Appendix Table 11.
Removing ground-truth action prompts reduces same-domain accuracy by only 2.39 percentage points on VP-UCF101 and 1.11 points on VP-HMDB51. Even without action-label prompting, Mask2Safe achieves 72.14% and 38.82%, exceeding TeD-SPAD by 9.56 and 8.17 percentage points, respectively.
Test-set prompt templates are given in Appendix C.3; evaluation protocols and results are in Appendix F. We will update Table 1 and the main paper (highlighted in blue) to integrate this test score. Our evaluation conclusions remain unchanged.
Q4. Independent Judge and Human Agreement
As discussed in Response 2, evaluation with open-source VLM judges yields consistent conclusions regarding SafeQA privacy and utility. We will report human evaluation on the generated benchmark questions once completed.
We thank the reviewers for the great suggestions and constructive feedbacks. Response to Weakness 1
We thank the reviewer for recognizing the value of localized replacement, improved anonymized-to-original transfer, and SafeQA's fine-grained privacy-utility evaluation. We appreciate the constructive questions and provide additional experimental evidence and methodological clarifications below along with the revised paper that will be updated shortly.
W1. Necessity of Learned Privacy Localization
Following the reviewer's suggestions, we provide ablations of the Privacy Masker and comparisons with simpler pseudo-label supervision and pretrained person segmentation (Full details in Appendix E, Tables 5-6, Figures 13-15).
Training objective ablations (Table 5).
We ablate the regional reconstruction losses, area regularization, and stop-gradient operations when training the Privacy Masker on Unsafe2Safe editing pairs. We also compare against models trained with differences between the unsafe-safe pairs on pixel level and DINO feature level as pseudo-targets. All variants are evaluated off-the-shelf against ground-truth privacy annotations on BIV-Priv-HIT (87 annotated videos), used here only as a reference given its mismatched privacy taxonomy and our coarse resolution masks.
Privacy Masker ablations and pseudo-label baselines on BIV-Priv-HIT (Appendix Table 5). Higher sensitive-part AUROC and Best F1 are better; [email protected] measures predicted mask coverage at threshold τ=0.5.
The auxiliary U-Net G provides a training-time reconstruction pathway that encourages the Privacy Masker to identify regions requiring replacement without spatial supervision. Although Lpref and Lpreserve supervise this pathway, their removal directly affects the learned masks. Without Lpref, masks become overly extensive in the absence of area regularization or collapse when it is retained. Removing Lpreserve also substantially degrades localization. Removing only the U-Net stop-gradient introduces artifacts on the token borders, whereas removing all stop-gradients collapses into almost black masks. These results demonstrate the importance of the regional reconstruction objectives and gradient constraints, without which the Privacy Masker will result in a collapse mode or an imperfect mask.
Both pseudo-label strategies perform substantially worse than the full model, indicating that simple label construction is insufficient to guide the learning of privacy localization. We found that such differences are sensitive to noise and changes unrelated to privacy. Additional implementation details and qualitative comparisons that visualize the artifacts are provided in Appendix E.
Comparison with object segmentation (Table 6). On COCO and Open Images, we evaluate class-wise pixel AUROC (distinguishing privacy-related objects from surroundings) and the selectivity gap (mean AUROC difference between privacy-related and non-sensitive control classes) against a person detector based on Mask R-CNN backbone.
Privacy discrimination and selectivity compared with pretrained object segmentation (Appendix Table 6). Higher privacy AUROC and selectivity gap are better.
Compared with person-only Mask R-CNN, which primarily responds to people, our Privacy Masker assigns substantially higher mask intensities to diverse privacy-sensitive objects than to non-sensitive categories. As shown in Appendix E.3 and Figure 15, mask intensity inside privacy-related objects is 2.24 times that of their surroundings on COCO (1.97 times on Open Images), compared with only 0.29 and 0.28 for non-sensitive objects, respectively. These results demonstrate that our learned masks capture privacy relevance beyond person detection.
Although SAM2 can propagate person masks across video frames, it does not resolve the limitations restricting to a predefined class and no semantic adapting. Our results, in contrast, support learning privacy-selective spatial representations from image-level supervision.