acceptodds
Withdrawn from ICLR 2027

PACO: Parsimonious Compositional Control for Instruction-Based Video Detoxification

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The growth of toxic videos that violate safety policies has become a critical challenge for large media platforms. With the emergence of instruction-based video editors, a rising solution is video detoxification, which removes toxic content from a source video while preserving its utility. Such editors are typically flow-matching models whose velocity fields are trained on source videos with editing instructions. At sampling time, the instruction guides the flow trajectory by modifying the velocity field along an editing direction. However, this procedure faces three challenges. First, single-direction guidance neglects the rich concept structure inherent to the velocity space. Second, determining which directions and edit strengths to apply is nontrivial, as a violation usually involves only a few specific toxic concepts. Third, existing video editing benchmarks rarely focus on safety, so a principled evaluation suite is lacking. To address these challenges, we propose Parsimonious Compositional Control (PACO), a video detoxification framework that optimizes a sparse composition of edit directions to steer the flow trajectory of the editor. For every video to be edited, we curate a dictionary of atomic edit directions in the velocity space to model the concept structure, and steer the editor with a sparse linear combination of the atoms whose coefficients optimize a joint safety, utility, and sparsity objective. Finally, we curate PACO-2K, a video detoxification benchmark of 2,450 task instances under 14 safety categories built from 27 textual guideline sources and 31 video datasets. Across multiple video editors, PACO consistently improves safety over steering baselines while preserving source video content well.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Video Detoxification, Video Safety, Representation Steering, Sparsity

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_C1tuOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

PACO works on video detoxification that removes harmful contents in a video with a frozen instruction-based video editor. For each video, an MLLM proposes 25 atomic edit instructions. Their CFG directions form a dictionary, and the editor is steered by a sparse linear combination of these directions. The coefficients are optimized per video with an evolution strategy. It also provides PACO-2K benchmark, consisting of 2,450 videos across 14 safety categories.

Critical strengths
  1. Detoxifying videos by editing, rather than filtering them out, preserves the amount of usable data, since harmful videos can be retained in a safe form instead of being discarded.
  2. The method is training-free and editor-agnostic, and the learned sparse coefficients are easy to inspect.
Critical weaknesses
  1. The problem formulation is crude with an unjustified assumption, and the method has very limited novelty. PACO assumes that summing the CFG directions of independently proposed instructions realizes the union of their edits, but this is supported only by analogy and never tested. The velocity field is nonlinear in the condition, and additive composition requires independent concepts, whereas detox atoms typically target the same region and interfere. Moreover, up to 25 directions, with coefficients as large as 15, are applied uniformly over space and time without any specific design or analysis. The paper's own non-optimized compositions (SEGA; MLLM-predicted coefficients) barely improve over plain CFG, so the coefficients act as a knob for black-box search rather than as interpretable edit strengths. Semantic harms such as misinformation or hate also fall outside the formulation. Finally, combining velocity fields under multiple conditions is well established, and a plain linear combination offers no novelty.
  2. Prohibitive cost and circular evaluation. Each video requires hundreds of full editor runs, each with 25+ guidance passes per step, which is several orders of magnitude more compute than a single edit, yet no runtime is reported. Moreover, the comparison is unfair. PACO directly optimizes the same judges used for most reported metrics, while all baselines are single-shot without judge feedback.
Questions for authors
  1. A detailed time statistics of the whole inference pipeline.
  2. How often does the 25-atom dictionary contain no edit that targets the toxic content?
  3. Can you provide evidence that CFG directions of different atomic edits compose additively in the editor's velocity field? For example, does applying two atoms jointly reproduce the effects of applying each one alone, especially when they target the same region or use large coefficients? If composition is not additive, in what sense are the optimized coefficients interpretable edit strengths, rather than parameters of a black-box search?
Reviewer_dzJJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents a methodology for video editing and a new video-editing benchmark, PaCo-2K. The benchmark is composed of a collection of pairs of toxic and safe videos targeting various toxicity-related topics (e.g., "Sex & Nudity", "Harassment & Bullying", "Intellectual Property"). The videos are short clips extracted from existing video datasets that target specific toxic concepts, and each pair is associated with a short sentence describing a broad edit that would make the video safer. The evaluation of the editing in terms of preservation and detoxification is left to two committees of AI-based judges, respectively.

The proposed method is an inference-time control method for applying fine-grained visual edits to videos. An MLMM reads the high-level instruction along with the video and suggests 50 (then refined to 25) visual edits to apply to make the video less toxic. The proposals are then embedded with an LLM into directions in the latent space and collected in a dictionary. These embeddings are then aggregated via a linear combination and used to compute the next flow-matching state. Iterating this step resolves into the final target video in K=20 steps. The vector of linear coefficients for the linear composition is selected after a zeroth-order optimization process to find their optimal values. This optimization step, selected after a comparison with alternatives, is designed as an Evolutionary Process guided by the loss-like signal defined by the judge evaluation and ends in R=50 steps.

The authors compare their proposal against 5 strong competitors on their proposed benchmark, consistently achieving the best detoxification results and competitive consistency scores. Since the proposed framework leverages the same judges used to compute the metrics, the authors also evaluate the candidates using a suite of held-out AI-based judges. Several further checks on the framework's effectiveness and validity are also available.

Critical strengths
  1. The paper is very clear and easy to follow, reporting many useful details to understand the methodology. Also, the experimental setup is well documented and rich in detail, which further helps in interpreting and assessing the results.
  2. The proposed benchmark encompasses several diverse toxicity-related topics and subtopics and can represent a useful resource for future research in safety-related steering.
  3. The results and the analyses reported show the effectiveness of the proposed framework.
Critical weaknesses
  1. While the text is clear and accurate, the figures generally are not. Many of them (Figs. 1, 2, 4, 5, 6, 8) are too small and very hard to read. I suggest cropping parts of the images (Figs. 4, 5, 6), reducing the number of examples shown (Figs. 4, 5), or rearranging them (Figs. 1, 2, 8) in order to improve readability.
  2. PaCo-2K has been automatically generated with the use of AI tools and provided with semantics via similarity scores with toxicity-centered prompts. This could affect quality when the footage is not selected well enough and compared with the assigned category. The paper does not provide enough assessment of the quality and coherence of the proposed resource.
  3. Following the experimental section, the Evolutionary strategy proposed by the authors seems to be very effective. However, it requires generating several edited videos to compute all the ∇^c(r)\hat{\nabla}_c^{(r)}. The paper does not report any analysis about the editing time required by their framework and competitors.
  4. The judges make their opinion upon 8 selected frames, which may not be representative enough, especially when it comes to assessing the consistency of the video, or with very long videos. The judges' sensitivity to this choice is not carefully checked.
  5. While the proposed benchmark is valuable in terms of toxic-topics coverage, it remains the only benchmark on which the authors compare their method with baselines. I also recommend testing PaCo and competitors on public benchmarks, e.g., those reported in Appendix B - "Video Editing Evaluation".
  6. The proposed paper reports very good metrics with the PaCo framework on PaCo-2K and releases a useful benchmark, which I think represents good points towards acceptance. On the other hand, I find the novelty in the framework limited. To my understanding, PaCo's main methodological contribution is the solution of the optimization problem of c (Sec. 3.3) with the Evolutive Strategy. This appears effective, but it could mask drawbacks and limitations that are not explicitly stated (e.g., Major Weakness 3). The limitations section (Appendix E) is very broad in scope. It does not address any structural or methodological limitations beyond a sentence about PaCo's weakness in editing long videos (which could be investigated more carefully given the existing time-length split; see Fig. 2).
Questions for authors
  1. How much time does the proposed framework take to make an edit on a single video? Does this time depend on the video length? Moreover, how much time do the competitors take to make the same edit? In general, it would be useful to have a breakdown of the computation time for each step of the proposed framework.
  2. Looking at the qualitative results reported, the framework seems biased towards generating light colors, cartoon-style videos when asked to remove toxicity. Is this true, and could it explain the relatively poor results in preservation?
  3. How does PaCo perform on editing benchmarks different from PaCo-2K?
  4. It is unclear to me where Sec. 3.1 should end. As long as this section describes PaCo-2K, I think all its content is intended to be part of the benchmark (e.g., the "atomic-edits" or the "Dictionaries"). Will the authors also release these lists and the dictionaries?
  5. As the authors report, they cannot distribute the videos from PaCo-2 K directly. However, they will "open-source an information directory with a set of documentation on how to access the publicly available version of each source". Could the authors please be more precise about this statement? In what form will they release their proposed benchmark?
Reviewer_Y1oTOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PACO, an inference-time control framework for video detoxification. It constructs a per-video dictionary of atomic edit directions in the velocity space of a frozen instruction-based video editor, and optimizes a sparse linear combination of these directions via zeroth-order optimization. The authors also introduce PACO-2K, a video detoxification benchmark.

Critical strengths

The proposed control scheme is relatively interpretable, as it optimizes a small number of atomic edit directions and makes the applied edits and their strengths controllable. The paper also presents a valuable video detoxification benchmark, PACO-2K, which covers 14 safety categories with 2,450 task instances and can support future research on safe video editing.

Critical weaknesses

W1:Missing key ablation on the dictionary.

There is no control experiment using a random dictionary or a dictionary without MLLM grounding, so it remains unclear whether the gains come from the MLLM-proposed grounded atomic edits or simply from the multi-direction sparse optimization pipeline.

W2:No inference cost analysis.

PACO requires per-video MLLM proposals and repeated rendering plus judge scoring during zeroth-order optimization, but the paper does not report wall-clock time, number of editor forward passes, judge calls, GPU cost, or cost relative to baselines. This makes practical deployment difficult to assess.

W3:Baseline selection does not match the task.

The paper frames the task as detoxification, but mainly compares against general video editing/steering methods rather than detoxification-specific or safety-oriented baselines. This is insufficient to establish the method’s advantage on the claimed task.

Questions for authors

see weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.