acceptodds
Under review as a conference paper at ICLR 2027

SIRUS: Training-Free T2V Concept Unlearning with Video-Aware Unlearning Evaluation

Abstract

Text-to-video (T2V) generators can synthesize realistic and temporally coherent videos, but controllably removing a target concept from a generator remains difficult. Unlike text-to-image concept erasure, T2V unlearning must suppress a target concept that may persist across frames while preserving non-target subjects, actions, scenes, and temporal structure. We propose SIRUS, a training-free inference-time framework for concept-level T2V unlearning. Given textual aliases of a target concept, SIRUS localizes target-related prompt evidence and suppresses target expression during sampling, without updating the text encoder or denoising network. We further introduce a video-oriented evaluation framework for T2V unlearning that separately measures target forgetting, non-target preservation, video quality, jailbreak robustness, and efficiency, using video-level failure criteria, frame-level residue statistics, paired preservation analysis, VBench-based quality diagnostics, and deployment overhead measurement. Across five safety, object, and style concepts on CogVideoX, SIRUS reaches 77.5% average forgetting success and 18.1% average frame hit, compared with 58.0% / 35.0% for VideoEraser, while reducing the average VBench quality drop from -0.040 to -0.011, yielding the strongest forgetting-quality trade-off among fully evaluated baselines. Transfer experiments on Wan2.2 further suggest that SIRUS generalizes across modern T2V backbones.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Text-to-Video Generation, Concept-Level Unlearning, Inference-Time Control, Diffusion Models, Safety Alignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_kyPNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SIRUS, a training-free method that removes a target concept from a text-to-video model at inference time. SIRUS projects selected prompt tokens off a subspace built from the concept's aliases and, early in denoising, subtracts a concept-restoring direction estimated from a branch conditioned on retrieved concept prompts. The paper also proposes VUEF, a video-level evaluation protocol that counts a video as failed when enough sampled frames show the target and scores preservation and quality. Experiments cover five safety, object and style concepts on CogVideoX with 200 videos each, against VideoEraser on all five and two further baselines on nudity only. SIRUS reaches 77.5% average forgetting success against 58.0% for VideoEraser, with VBench video quality of 0.808 against 0.778 (0.818 for the unmodified model).

Critical strengths
  1. The sampling-side correction is effective. Prompt editing alone can leave the concept recoverable from the denoiser's priors, so SIRUS subtracts a restoring direction estimated from a branch conditioned on retrieved concept prompts.
  2. The forgetting gain over VideoEraser is large and consistent. On 1,000 prompt-matched CogVideoX video pairs, average success is 77.5% against 58.0%, and the gap's 95% confidence interval is 16.0 to 23.0 points (Table 12). SIRUS leads on every concept and every failure threshold of 1 to 16 frames (Table 11).
Critical weaknesses
  1. The method is never evaluated on prompts without the target concept. When no alias matches and every token's similarity to the concept subspace is below 0.25, the trigger still selects up to three tokens and applies its strongest settings (Tables 21 and 22). The claimed precision on benign prompts is therefore untested, although the paper itself says this case requires checking collateral effects.
  2. The component ablation is thin and partly inconsistent. It covers only nudity, defines neither the variants nor the sample size, and its full-method success (80.5%) differs from the main nudity result (84.5%, Table 1). Removing the token trigger raises person retention from 78.5% to 81.4% and lowers non-target semantic change (CSDR) from 10.0 to 8.2, so its preservation benefit is unclear (Table 4).
Questions for authors

None.

5 comments
AuthorsOpenReview ↗

Reply to Reviewer kyPN

We appreciate your feedback and respond to the two concerns you raised in W1 and W2.

W1. Retention on unrelated prompts. The soft fallback can intervene even when no alias matches and all token similarities are below 0.25. We generated 50 videos per requested concept with the original CogVideoX. For each of the four other removal targets, we ran SIRUS with fallback on and off using the same prompts and seeds. Each cell gives the counts retained with fallback on / retained with fallback off / present in the original videos. For instance, 39 original videos showed nudity. When SIRUS targeted churches, 34 of those 39 still showed nudity with fallback on and 33 with it off.

Requested conceptOriginal CogVideoX videos showing the requested conceptRemove nudityRemove churchRemove garbage truckRemove parachuteRemove Van Gogh style
Nudity39/50—34/33/3932/32/3932/32/3933/34/39
Church45/5044/44/45—44/44/4545/44/4544/44/45
Garbage truck46/5045/43/4645/45/46—45/45/4644/45/46
Parachute47/5045/46/4746/46/4746/46/47—46/46/47
Van Gogh style32/5025/24/3225/26/3225/25/3224/25/32—

On prompts requesting the removal target, we also tested 50 videos per concept. Each cell gives results with fallback on/off.

TargetVideo forgetting (%)Frame forgetting (%)LPIPSCLIP preservation ratioVBench quality
Nudity84.0/72.088.6/77.20.669/0.6410.971/0.9790.812/0.805
Church64.0/52.072.1/58.90.668/0.6331.033/1.0450.831/0.834
Garbage truck80.0/70.084.3/76.10.671/0.6421.052/1.0290.816/0.818
Parachute60.0/54.070.5/63.60.594/0.5591.039/1.0450.785/0.781
Van Gogh style98.0/96.098.6/96.80.742/0.7111.002/1.0140.847/0.842
FallbackUnrelated prompts: conditional retentionTarget prompts: video forgettingTarget prompts: frame forgetting
On769/836 (92.0%)193/250 (77.2%)82.8%
Off769/836 (92.0%)172/250 (68.8%)74.5%

When the prompt requested the removal target, fallback improved forgetting for all five concepts, yielding 21 more successful removals across 250 videos. When the prompt requested a different concept, the original video showed that concept in 836 comparisons spanning a person’s state, objects, and an artistic style. SIRUS retained the requested concept in 769/836 comparisons (92.0%) with either setting. Enabling fallback therefore improved target removal with no observed drop in overall retention on unrelated prompts.

W2. Ablation batches and the trigger trade-off. Full SIRUS succeeded on 169/200 nudity videos (84.5%) in the main evaluation (Table 1) and on 161/200 (80.5%) in the separate ablation batch (Table 4). Every variant in Table 4 was evaluated on that same 200-video ablation batch.

On the ablation batch, the trigger increased successful removals from 110/200 to 161/200 (55.0% to 80.5%) and reduced frame hit from 34.4% to 15.9%. As you observed, disabling the trigger improved person retention from 78.5% to 81.4% and reduced CSDR from 10.0 to 8.2. Each CSDR value is an average over all 200 cases, including videos in which nudity remained. Simply copying an original video that contains nudity would give both LPIPS and CSDR their ideal value of zero while preserving nudity itself. To assess which variant better preserves other content after removing nudity, the relevant comparison is on matched cases where both variants succeeded.

We extended the ablation to the other four targets using 50 videos per concept.

TargetVariantVideo forgetting (%)Frame forgetting (%)LPIPSCLIP preservation ratioVBench quality
ChurchFull64.072.10.6681.0330.831
Churchw/o reference22.041.50.6770.9850.812
Churchw/o trigger36.050.00.6811.0210.819
Churchw/o projection44.056.30.6990.9680.843
Garbage truckFull80.084.30.6711.0520.816
Garbage truckw/o reference34.050.30.6651.0320.814
Garbage truckw/o trigger52.064.60.6991.0150.812
Garbage truckw/o projection58.068.80.6820.9730.815
ParachuteFull60.070.50.5941.0390.785
Parachutew/o reference26.041.90.5721.0390.786
Parachutew/o trigger38.053.10.5991.0020.761
Parachutew/o projection44.057.40.6210.9440.773
Van Gogh styleFull98.098.60.7421.0020.847
Van Gogh stylew/o reference62.071.80.7211.0140.842
Van Gogh stylew/o trigger80.087.50.7391.0420.825
Van Gogh stylew/o projection74.082.10.7650.9800.856

Across these four targets, the trigger raised mean video forgetting by 24 percentage points (51.5% to 75.5%). Full SIRUS also exceeded the variant without projection (55.0%) and the variant without the reference branch (36.0%). Removing the reference branch caused the largest drop on each of these four targets (34–46 percentage points). The trigger improved forgetting across all five concepts.

Dear Reviewer kyPN, We are following up on our response to the concerns you raised. We have provided retention results with fallback enabled and disabled, extended the component ablation to the other four concepts, and clarified the separate evaluation batches behind Tables 1 and 4. We also discuss the trigger’s effect on both forgetting and preservation. If you have had a chance to review these additions, we would appreciate hearing whether they address your concerns and which points would benefit from further clarification.

Thank you for the thorough rebuttal and the additional results. I remain concerned about SIRUS's precision and practical usefulness. When other concepts are removed, only 24–26 of the 32 Van Gogh-style videos keep the style, with fallback on or off, which is a real issue in controllable video generation.

Reviewer_4vvSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies concept unlearning in text-to-video generation. The method SIRUS uses a concept subspace to identify and edit target-related text embeddings. It then uses a constrained concept residual to suppress the target concept during diffusion sampling. The paper also introduces VUEF to assess forgetting, content preservation, video quality, robustness, and efficiency.

Critical strengths

The method is clearly presented and easy to follow. It requires no fine-tuning, making it practical to apply to existing video generators. Compared with VideoEraser on CogVideoX, SIRUS improves both concept forgetting and video quality. The evaluation also checks whether other content, including people, remains after concept removal.

Critical weaknesses
  1. The benefit of reference retrieval needs a clearer test. VideoEraser already uses concept-conditioned guidance. I would like to see a comparison between retrieved references and a fixed concept reference, with the rest of the pipeline unchanged. This would show how much retrieval itself contributes to the improvement.
  2. It is unclear whether SIRUS changes videos that do not need concept removal. For example, when configured to remove garbage trucks, does it affect a prompt about a person walking in a park? The authors should test such prompts and report whether the method activates or changes the requested content. Testing with the fallback mechanism turned off would help identify unnecessary edits.
Questions for authors
  1. Does reference retrieval improve forgetting or video quality compared with a fixed concept reference under the same settings?
  2. How does SIRUS affect prompts that do not involve the target concept? Does turning off the fallback reduce unintended changes?
4 comments
AuthorsOpenReview ↗

Reply to Reviewer 4vvS

Thank you for your review. Below, we respond to your questions about the reference bank and the fallback mechanism.

W1/Q1. Does the reference bank affect forgetting or quality? By default, SIRUS builds a candidate bank for each prompt by paraphrasing its target expression and expanding the variants with templates. It retrieves and combines the top four available candidates. We tested whether a larger shared bank could match the per-prompt bank. All prompts for a given concept shared the same 16 candidates in one setting and the same 32 in the other. We evaluated 50 videos per concept and setting.

TargetBankVideo forgetting (%)Frame forgetting (%)LPIPSCLIP preservation ratioVBench quality
NudityPer prompt84.088.60.6690.9710.812
NudityShared 1666.073.90.6720.9750.809
NudityShared 3272.079.40.6700.9740.811
ChurchPer prompt64.072.10.6681.0330.831
ChurchShared 1650.062.50.6651.0450.832
ChurchShared 3254.066.30.6641.0370.830
Garbage truckPer prompt80.084.30.6711.0520.816
Garbage truckShared 1674.080.50.6771.0460.812
Garbage truckShared 3276.081.80.6721.0490.810
ParachutePer prompt60.070.50.5941.0390.785
ParachuteShared 1648.059.90.5921.0390.786
ParachuteShared 3252.063.00.5991.0420.791
Van Gogh stylePer prompt98.098.60.7421.0020.847
Van Gogh styleShared 1694.096.10.7431.0010.848
Van Gogh styleShared 3296.096.90.7381.0040.848

Mean video forgetting was 77.2% with the per-prompt bank, compared with 66.4% for Shared 16 and 70.0% for Shared 32. Frame forgetting followed the same pattern (82.8%, 74.6%, 77.5%). Mean VBench quality was comparable across the three settings (0.8182, 0.8174, and 0.8180, respectively).

Doubling the shared bank from 16 to 32 candidates improved mean video forgetting by 3.6 percentage points. Even with 32 candidates, the gap to the per-prompt bank was 12 points for nudity and 10 for church, compared with 4 for garbage trucks and 2 for Van Gogh style. This pattern suggests that a fixed bank is harder to use effectively when prompts describe the same concept in more varied ways. Across all five concepts tested here, tailoring the reference bank to the prompt improved both video and frame forgetting over either shared bank, with comparable mean VBench quality.

W2/Q2. What changes when fallback is disabled? Each row is a concept the prompt asks CogVideoX to generate. Each removal column is a different concept SIRUS is set to suppress. We tested all 20 pairs, reusing each row’s 50 original videos across four removal targets. Read each cell as retained with fallback on / retained with fallback off / present in the original videos. For example, 34/33/39 means that the original model generated nudity for 39 of the 50 nudity prompts. When SIRUS was set to remove churches, nudity remained detectable for 34 of those 39 prompts with fallback on and 33 with it off. All three runs used the same prompt and random seed for each case.

Requested conceptOriginal CogVideoX videos showing the requested conceptRemove nudityRemove churchRemove garbage truckRemove parachuteRemove Van Gogh style
Nudity39/50—34/33/3932/32/3932/32/3933/34/39
Church45/5044/44/45—44/44/4545/44/4544/44/45
Garbage truck46/5045/43/4645/45/46—45/45/4644/45/46
Parachute47/5045/46/4746/46/4746/46/47—46/46/47
Van Gogh style32/5025/24/3225/26/3225/25/3224/25/32—

With garbage-truck removal enabled, both fallback settings retained churches in 44/45 cases and parachutes in 46/47. Across all 20 pairs, there were 836 comparisons in which the original video showed the requested concept. That same concept was detected in 769 of the matched SIRUS outputs (92.0%) with fallback on, and also in 769 (92.0%) with it off.

We then tested forgetting on 50 prompts per concept that requested the concept SIRUS was set to remove. Each cell reports results with fallback enabled / disabled.

TargetVideo forgetting (%)Frame forgetting (%)LPIPSCLIP preservation ratioVBench quality
Nudity84.0/72.088.6/77.20.669/0.6410.971/0.9790.812/0.805
Church64.0/52.072.1/58.90.668/0.6331.033/1.0450.831/0.834
Garbage truck80.0/70.084.3/76.10.671/0.6421.052/1.0290.816/0.818
Parachute60.0/54.070.5/63.60.594/0.5591.039/1.0450.785/0.781
Van Gogh style98.0/96.098.6/96.80.742/0.7111.002/1.0140.847/0.842
FallbackTarget prompts: video forgettingUnrelated prompts: conditional retention
On193/250 (77.2%)769/836 (92.0%)
Off172/250 (68.8%)769/836 (92.0%)

With fallback enabled, SIRUS successfully removed the target in 21 more videos, and frame forgetting rose from 74.5% to 82.8%. Fallback improved forgetting for every tested target while preserving the same proportion of unrelated requested concepts overall (92.0%).

Dear Reviewer 4vvS, Thank you again for your feedback. Our response includes comparisons with shared reference banks and experiments with fallback enabled and disabled. The per-prompt bank improved forgetting across all five concepts, while enabling fallback increased overall forgetting success from 68.8% to 77.2%, with overall retention on unrelated prompts at 92.0% under both settings. We would appreciate the opportunity to hear your thoughts on these results and discuss any questions you may still have.

Thank you for the detailed response and additional experiments, which address my main questions. I will keep my current score.

Thank you for taking the time to review the additional experiments. We’re glad they addressed your main questions, and we appreciate your feedback.

Reviewer_JV2bOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a method to perform training-free T2V Unlearning, smoothing and controlling the effect of concept embeddings during the diffusion process.

The argument supporting the work starts with the idea that concept erasure rarely localize in a single token in the prompt used for text conditioning, thus context matters. Moreover, simple pharaprasing can trick a method based on simple word matching.

Starting from this understanding, the paper present a method that can be roughly divided into three parts. First, it constructs a concept subspace using embedded aliases, from which a representative basis B is extracted. With this basis a concept-aligned operator can be defined for further use. Second, to address concerns regarding concept context and rewording, a hierarchical trigger is defined using first exact word matching, then strong similarity and as a fallback strict/soft top-k. This produces a profile that defines hyperparameters for the unlearning: the first triggers requires less strength because the concept is already strongly present and easy to suppress, while later triggers requires more strength beacuse the concept is more nuanced. (Appendix C.3) Triggers not only define the hyperparameters for the unlearning, but also the mask of tokens involved by this removal. Using this mask and values, the method suppress the critical tokens, obtaining an edited condition. A similar but different approach is used to compute a concept reference condition that define target restoration. Finally, all the branches here presented coverge into a momentum-based modification to the classifier-free guidance to move away from the target concept without destroying generation quality and semantically unrelated informations.

The paper also present an evaluation framework, VUEF, spanning different axis like forgetting, preservation, video quality, robustness and efficency. Using concepts, objects and styles the method compare with SOTA techniques and video backbones, like CogVideoX.

Critical strengths
  • The motivation is clear and supports the methodological framework: context and semantic similarity are important clues for concept unlearning and must be addressed.
  • The evaluation framework addresses many concerns related to general visual unlearning including preservation, forgetting and overall quality
  • Table 4 shows that the different components employed contribute to the method’s performance in this setting, with an important ablation study.
Critical weaknesses
  1. While it is clear that an unlearning method requires testing over broad categories, it is unclear why the paper presents 'nudity' as a category and single objects as evaluation targets by themselves, and not inside a representative category such as 'objects'
  2. The positive-concept reference branch uses a bank of prompts whose creation is not sufficiently explained.
  3. An important part of unlearning is the trade-off between preservation and removal. The paper does not report any metric when the unlearned model is prompted with concepts unrelated to the target concept.
  4. The generalization claims in Section 5.4 are limited, the method is evaluated on only two backbones. Comparisons with 'Refusal Vector' use a different backbone.
Questions for authors
  1. Nudity is a broad concept, with many different interpretations that can generate unsafe videos. If possible for that category, can the paper present an evaluation category for multiple semantically related objects? Like 'Trucks' for the 'Garbage Truck'.
  2. How is the prompt bank for the positive concept-reference branch constructed?
  3. Is it possible to present a study for the retaining capabilities of the unlearned model when prompted on unrelated concepts? (e.g. Unlearned model for 'nudity' prompted about 'Garbage Truck')
  4. The generalization claim would be stronger with an evaluation on at least one additional backbone. Is it possible to extend Table 5 with OpenSora2, the backbone used for 'Refusal Vector'?
2 comments
AuthorsOpenReview ↗

Reply to Reviewer JV2b

Thank you for your review. We address your four questions in order below.

W1/Q1. Why these concepts, and what happens to related objects? We chose the targets following the taxonomy in Liu et al., Rethinking Machine Unlearning in Image Generation Models (CCS 2025), which classifies targets by spatial extent (global or local) and perceptual attributes (abstract or concrete). “Van Gogh style” is global and abstract because it affects the appearance of the whole frame. “Nudity” is local and abstract because it describes a person’s state. Here the aim is to remove nudity while preserving the person. “Church”, “parachute”, and “garbage truck” are local, concrete entities. These three targets occupy the same category in the taxonomy. Reporting each separately shows how forgetting varies within that category.

With “garbage truck” removal enabled, SIRUS generated a truck for 46/50 prompts requesting “a truck” (92%). It could thus still generate trucks when targeting this specific subtype.

W2/Q2. How is the reference prompt bank built? We start with the target expression in each input prompt, paraphrase it, and expand each variant using five templates. We use the expression on its own or in “a photo of …”, “a video of …”, “a close-up of …”, and “footage of …”. For example, “garbage truck” can be paraphrased as “refuse truck”, giving candidates such as “a close-up of a refuse truck”. We remove duplicates, encode the candidates, and rank them by cosine similarity to the selected tokens in the input prompt. We combine up to four of the highest-ranked candidates, weighting each by its similarity score. The templates are shared across prompts, while the target wording in each prompt determines which expressions enter its bank and which references receive the most weight.

W3/Q3. What happens when the prompt requests an unrelated concept? On the same 50 garbage-truck prompts with matched seeds, CogVideoX and SIRUS with nudity removal each generated a detectable garbage truck in 46/50 videos. SIRUS retained the truck in 45 of the 46 cases where CogVideoX generated one (97.8%). The table repeats this test for all 20 ordered pairs. The row names the concept requested in the prompt, and the column names the concept SIRUS is set to remove. We counted a concept as present if it appeared in at least 4 of 16 sampled frames.

Requested conceptOriginal CogVideoX videos showing the requested conceptRemove nudityRemove churchRemove garbage truckRemove parachuteRemove Van Gogh style
Nudity39/50—34/3932/3932/3933/39
Church45/5044/45—44/4545/4544/45
Garbage truck46/5045/4645/46—45/4644/46
Parachute47/5045/4746/4746/47—46/47
Van Gogh style32/5025/3225/3225/3224/32—

Of the 250 videos generated by the original model (50 per requested concept), 209 showed the requested concept. We tested each of those 209 prompts against four other removal targets, giving 836 comparisons. For prompts requesting a church, garbage truck, or parachute, the requested object remained detectable in 539/552 comparisons (97.6%). Across all five requested concepts, spanning a person’s state, objects, and an artistic style, SIRUS retained the requested concept in 769/836 comparisons (92.0%) while targeting a different one.

W4/Q4. How does SIRUS compare with Refusal Vector on OpenSora2? We extended the CogVideoX and Wan2.2 evaluation to a third backbone, OpenSora2, using 200 videos per concept. This lets us compare SIRUS directly with the OpenSora2 results for Refusal Vector reported in the submission. Each cell gives video forgetting / frame forgetting (%) using the same 16-frame protocol.

TargetRefusal VectorSIRUS
Nudity33.0/40.382.5/84.6
Church25.5/31.467.0/75.1
Garbage truck16.0/20.275.0/79.6
Parachute12.0/16.153.5/61.4
Van Gogh style98.0/98.497.5/97.8
Mean36.9/41.375.1/79.7

Across the five concepts, video forgetting success was 75.1% for SIRUS and 36.9% for Refusal Vector, a gain of 38.2 percentage points. Frame forgetting improved by 38.4 points (79.7% vs. 41.3%).

We measured preservation relative to the original OpenSora2 videos and averaged quality over five VBench dimensions.

OpenSora2 methodLPIPS ↓CLIP preservation ratioVBench quality ↑
Original model——0.779
Refusal Vector0.5000.9630.767
SIRUS0.6051.0260.768

Refusal Vector’s lower LPIPS comes with less complete forgetting. When the target remains visible, the video can stay perceptually closer to the original. We therefore read LPIPS together with forgetting success, non-target CLIP alignment, and video quality. On this third backbone, SIRUS more than doubled mean video forgetting success relative to Refusal Vector (75.1% vs. 36.9%), while maintaining non-target CLIP alignment close to the original (ratio 1.026) and comparable VBench quality (0.768 vs. 0.767).

Dear Reviewer JV2b, We would like to follow up on our response to your review. We have included the truck example you suggested, retention results on unrelated prompts, and a direct comparison with Refusal Vector on OpenSora2. On this additional backbone, SIRUS achieved 75.1% mean video forgetting success, compared with 36.9% for Refusal Vector, with comparable video quality. When you have a chance, we would appreciate your thoughts on whether these results address your questions. We would be happy to clarify any remaining points.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.