- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes StreamGuard, a real-time, policy-following video guardrail that combines streaming frame-level detection, event-level summarization, parallel inference, and multi-level policy alignment. The authors also introduce two benchmarks, Safe2Shot and AdvVideo-Bench, to evaluate fleeting unsafe content and adversarial robustness. Experiments demonstrate improvements in detection performance and inference efficiency.
Critical strengths
-
Important and practical problem. The paper addresses real-time video safety monitoring, particularly the detection of short-lived unsafe content that conventional frame-sampling approaches may miss.
-
Interesting streaming architecture. StreamGuard combines frame-level detection, event-level reasoning, and asynchronous parallel inference to improve detection efficiency.
Critical weaknesses
-
Inconsistent frame-exact coverage claim. The paper claims frame-exact coverage without sampling loss, but StreamGuard processes videos at only 2 FPS. Consequently, unsafe events lasting less than 0.5 seconds may still be missed, contradicting the motivation of detecting fleeting unsafe content.
-
Potentially unfair frame-budget comparison. StreamGuard processes videos at 2 FPS, while other baselines receive only 16 uniformly sampled frames. Appendix B.2 shows that StreamGuard's performance decreases substantially under the same 16-frame budget, raising concerns about the fairness of the comparison.
-
Unclear computational overhead of parallel inference. The reported inference speedup relies on multiple parallel workers, but the paper does not clearly specify whether additional GPU resources are required. GPU utilization, memory consumption, and efficiency under fixed hardware budgets should be evaluated.
-
Limited cross-generator generalization. The paper does not sufficiently evaluate whether StreamGuard generalizes to unsafe videos produced by different video generation models. A leave-one-generator-out evaluation would strengthen the generalization claim.
-
Potential dependence on attack-specific training distributions. TV2T and TV2V use similar attack construction pipelines for training and testing. The reported robustness improvements may therefore reflect learned attack-specific patterns rather than generalization to unseen attacks.
-
Insufficient evaluation of policy-following robustness. The improvements may largely result from adversarial training rather than stronger policy-following capabilities. More evaluations on unseen safety policies and separate visual-obfuscation and prompt-injection attacks are needed.
Questions for authors
-
Frame coverage. How does StreamGuard handle unsafe events shorter than 0.5 seconds when processing videos at 2 FPS?
-
Comparison fairness. Could the authors compare StreamGuard with baselines under identical frame and computational budgets?
-
Inference efficiency. Does parallel inference require additional GPUs, and how does the speedup change under a fixed GPU budget?
-
Cross-generator generalization. How well does StreamGuard detect unsafe videos generated by unseen video generation models?
-
Unseen adversarial attacks. Could the authors evaluate robustness against attack templates and transformations not included in training?
-
Policy-following robustness. How much of the improvement comes from policy alignment rather than adversarial training?
4 comments
Response to Reviewer 3rMJ (Part 2/3)
W3 / Q3. Do parallel workers need extra GPUs? Not in any experiment reported in the paper, but in the current implementation, yes, beyond four concurrent summaries. All latency numbers in the paper (Table 7, Figure 3) were measured on a single NVIDIA H100. In that implementation, one labeling process holds the full model, and summary workers are separate processes on the same GPU, allocated on demand when an event enters its explanation phase. Each summary worker holds its own copy of the weights, so memory grows by roughly 15 GiB per active worker. The measured peak GPU memory (nvidia-smi), reached with the labeling process plus 4 concurrent summary workers, is 79 GB, close to the 80 GB capacity of the H100. The “unlimited workers” curve in Figure 3 corresponds to this on-demand policy, i.e., the “sufficient workers” case of Eq. (9). In practice at most 5 processes (1 labeling + 4 summary workers) were ever active at once, including for the "unlimited workers" curve in Figure 3, which is measured rather than simulated: events are short (4.36 frames on Safe2Shot, 14.63 on the B.7 livestreams), so a worker becomes idle as soon as its summary is finished. In the Table 7 run (2.14 s), between 2 and 5 processes were active at a time (1 labeling + 1–4 summary workers).
With 4 summary workers the GPU is nearly full, so if a fifth concurrent summary were needed, the current implementation would place it on an additional GPU. This did not occur in any reported experiment, but it does mean the multi-process design does not scale to higher concurrency (e.g., several streams per GPU) on one card. We agree that per-process model copies are memory-inefficient, and the structure of StreamGuard allows a leaner design in which a worker is a KV-cache sequence slot on a single copy of the weights, served with continuous batching:
- Closure precedes the summary. An event is closed as soon as a frame label is emitted without
<continue>(Appendix A.3), before the summary is generated, so the next event can start immediately without speculative execution. - No copying. The summary keeps decoding in the closing event’s slot; the next event opens a new slot.
- Shared prefix. Every event restarts from the same policy header H(P, q) (§3.2), whose KV cache is computed once and shared across events and streams, so starting an event costs no prefill.
- Small, short-lived state. With the InternVL3-8B base model, whose LLM is Qwen2.5-7B (28 layers, 4 KV heads, head dim 128, fp16), KV is about 56 KB per token; an event context is about 1,100–1,400 tokens (about 80 MB) on Safe2Shot and about 4,000 tokens (about 220 MB, 14.63 frames per event) on the B.7 livestreams, and it is released once its summary is done. Hundreds of slots fit on one GPU.
Summary decoding and the next event’s labeling are merged into the same forward passes. Because decoding is memory-bandwidth bound, the per-step cost grows only slowly with batch size.
We are implementing this single-copy design, and in the final version we will compare it with the multi-process implementation on the same H100 (latency, throughput and peak memory). We now state the hardware configuration in a new appendix section and the memory cost of the current multi-process design as a limitation.
Response to Reviewer 3rMJ (Part 3/3)
W4 / Q4. Unseen video generators. TV2V samples in AdvVideo-Bench are produced by three generators: CogVideoX (474), Wan 2.1 (912) and Cosmos (514), 1,900 in total. The phrase “we fix a generator G” in Appendix A.7 describes the per-sample procedure and was misleading; we now name all three generators in §4.2 and A.7. This makes a leave-one-generator-out evaluation possible: retrain without one generator, test on all of its videos (474 / 912 / 514), and compare with a control that removes the same number of randomly chosen TV2V videos, so that the gap isolates the cost of an unseen generator. We will add this in the final version. In the meantime, the MCV and SPTV results (response to egus-W3) show StreamGuard's behavior on attack videos built entirely outside our data pipeline.
W5 / Q5. Unseen attack templates and transformations. We agree that the original TV2T/TV2V test sets are in-distribution, as noted in §5.1, and that AutoDAN (B.6) is not truly unseen because AutoDAN-style data appear in training. We therefore evaluate on two attacks from other authors whose construction pipelines are absent from our training data: SPTV (CVPR 2026) and MCV (ACL 2026), using their public releases. StreamGuard misses 19.6% of SPTV attacks and 23.4% / 34.4% of explicit / implicit MCV attacks, rising to 27.5% / 40.0% for 4-clip attacks (details in the response to egus-W3). Leave-one-out retraining over TV2T transformation families and TV2V template families will be added in the final version.
W6 / Q6. Policy alignment vs. adversarial training. Table 4 separates the two factors on AdvVideo-Bench. Removing the adversarial data costs 10 points on both TV2T and TV2V (76.0 → 66.0, 68.0 → 58.0). On these in-distribution attacks the adversarial data is therefore the main source of robustness, and without it StreamGuard would fall below SafeWatch (69.0 / 62.0). Policy alignment contributes on top of it: with the adversarial data kept, removing GRPO costs 4.0 / 5.0 points, and replacing the structured policy with a simple prompt costs 2.0 / 4.0 points. On attacks built outside our data pipeline (MCV, SPTV; response to egus-W3), StreamGuard still flags most attack videos. We will add the full 2×2 ablation, evaluated on these unseen attacks, in the final version.
Thank you again for your critical and constructive review. As the discussion period is coming to an end, we would like to check whether our responses have addressed your concerns, in particular those about soundness. In brief:
Frame-exact claim (W1). The wording has been corrected to "dense, uniform-rate coverage of every ingested frame", and sub-0.5 s events are listed as a limitation (Appendix C). Frame budget fairness (W2). At matched cost, StreamGuard reads the full 2 FPS stream faster (3.97 s serial, 2.14 s parallel) than every baseline reads 16 frames. This includes its same-size base model InternVL3-8B (4.05 s), so the gain comes from the streaming design and not from more compute. The inaccurate "comparable" description in Appendix B.2 has been corrected. GPU resources (W3). All latency results are measured on a single H100 with at most four concurrent summary workers (peak 79 GB). The configuration and its memory cost are stated in Appendix A.11 and Appendix C. Generators and unseen attacks (W4/W5). The three TV2V generators are now named in §4.2 and Appendix A.7. Appendix B.9 (Table 18) adds an evaluation on two external video jailbreaks, MCV and SPTV, whose construction pipelines are absent from our training data. Policy alignment vs. adversarial training (W6). We now state explicitly in §5.5 that the adversarial data is the main source of robustness on the in-distribution attacks; policy alignment adds 2–5 points on top of it. We would be glad to answer any remaining questions before the discussion closes. If our responses address your concerns, we would be grateful if you would consider revisiting your soundness assessment and score.
Response to Reviewer 3rMJ (Part 1/3)
We thank the reviewer for the detailed and critical review. We understand the core concern to be that the gains might come from more frames and in-distribution adversarial training rather than from the method itself. We address each point below with existing controlled results and new experiments, several of which are shared with other reviewers (General Response G1–G3).
W1 / Q1. Events shorter than 0.5 s at 2 FPS. The reviewer is right that “frame-exact coverage without sampling loss” overstates what a 2 FPS system does; we have replaced it with “dense, uniform-rate coverage of every ingested frame.” Three clarifications:
W2 / Q2. Identical frame and compute budgets (G1). We should not have described the 16-frame result in B.2 as "comparable": at 16 frames StreamGuard is below GPT-4.1, as one would expect from an 8B model. The comparison we argue for is at matched cost. StreamGuard reads the full 2 FPS stream in 3.97 s (2.14 s parallel), below every 16-frame baseline in Table 7, including its same-size base model InternVL3-8B (4.05 s). This shows that dense coverage comes from the streaming design, not from model size. On identical frames it leads the best baseline by 3.1 frame F1. Open baselines at 32/64 frames and the full accuracy–latency frontier will be added in the final version.