- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces CTASO, a black-box framework for generating content-preserving adversarial style transformations against guarded VideoLLM pipelines. CTASO constructs a continuous style space, applies temporally coordinated video editing, restores salient regions and text from the source video, and uses feasibility-first CMA-ES optimization to search for transformations that bypass an input guard. Experiments cover three content guards and four downstream VideoLLMs, with human auditing used to verify event preservation.
Critical strengths
- The paper identifies an important attack surface by distinguishing input-guard evasion from downstream VideoLLM jailbreak success.
- The evaluation is relatively comprehensive, covering multiple guards and VideoLLMs, with human auditing and component ablations to verify event preservation and method effectiveness.
Critical weaknesses
- The computational efficiency of CTASO is not adequately evaluated. The paper reports the number of candidate evaluations before the first successful attack, but this is not equivalent to time complexity or practical runtime. Each CTASO candidate requires diffusion-based video editing, saliency restoration, OCR processing, multiple similarity measurements, and possibly a guard query. Moreover, the reported mean candidate count is conditioned on successful runs, making direct efficiency comparisons potentially misleading. The paper does not report wall-clock time, GPU hours, memory consumption, or runtime scaling with video length and resolution, nor does it compare these costs with prior attacks. Such measurements are important for evaluating applicability in realistic settings.
- The black-box threat model assumes access to unusually informative guard feedback. CTASO requires both the guard decision and a continuous unsafe-confidence score derived from next-token logits. In many realistic systems, an external attacker receives only a binary allow/block decision and cannot access logits or calibrated confidence values. Thus, the proposed setting is stronger than a conventional decision-only black-box threat model, and it is unclear whether CMA-ES remains effective when only binary feedback is available.
- The preservation constraints do not directly model temporal event semantics. The automatic constraints rely primarily on frame-level CLIP similarity, salient-region correspondence, Sobel structure, and OCR consistency. These measurements can preserve appearance while failing to preserve action order, object interactions, causal relationships, or other temporally defined aspects of an event. Human audit reduces evaluation errors but is applied only after optimization and therefore cannot correct a misaligned search objective. This concern is amplified by the evaluation protocol in which the guards and VideoLLMs observe only four or eight sampled frames, potentially underrepresenting longer or transient video events.
Questions for authors
- Can the authors report wall-clock runtime, GPU usage, peak memory, and scaling with video duration and resolution for CTASO and the main baselines? How many diffusion-editing operations and guard queries are required when failed runs are also included?
- How does CTASO perform when the attacker receives only a binary allow/block response? Can the method be adapted to a strictly decision-only black-box setting without unsafe-confidence or logit access?
- Can the authors incorporate or evaluate temporally semantic preservation criteria, such as action recognition or video-level event representations? How sensitive are the results to the number and positions of sampled frames, particularly for long videos or short transient events?