acceptodds
Under review as a conference paper at ICLR 2027

UniRGBA : Unifying RGB and Alpha in Pretrained Video Diffusion Models

Abstract

Native alpha is essential for compositing visual effects, yet video diffusion models remain RGB-only and matting pipelines underrepresent continuous transparency. We present UniRGBA, an interface-preserving extension of a pretrained video diffusion model for native RGBA video. UniRGBA-VAE expands the Wan2.1 3D causal VAE from RGB to RGBA by zero-initializing the new boundary channels and updating only symmetric 2+2 layers (about 5% of VAE parameters), while retaining the original 16-channel latent interface. A KL-anchored two-stage objective learns alpha structure and sharp compositing details; standard DoRA then adapts the shared Wan2.1 DiT for RGBA T2V and I2V. On a held-out visual-effect benchmark, UniRGBA-VAE improves PSNR from 25.00 to 29.16 over a retrained Wan-Alpha baseline (+4.16dB), including +5.33dB RGB and +3.95dB alpha gains. RGBA T2V improves text alignment (+0.28), naturalness (+0.33), and temporal flickering (+0.0050) over Wan-Alpha, while RGB transfer exposes a measurable -0.095 Dynamic Degree cost. These results show that native alpha can be added through a small VAE-side intervention while retaining a reusable latent interface.

Primary area:
generative models
Keywords:
native RGBA video generation, alpha-aware video diffusion, transparent video synthesis, latent video autoencoders, parameter-efficient adaptation, text-to-video and image-to-video generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_JB8pOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to extend a pretrained RGB video generator to produce transparent RGBA videos while retaining its RGB generation capability. It adapts the causal video VAE at its input and output boundaries, preserves the existing 16-channel latent interface, and uses lightweight DoRA tuning for downstream generation. The paper evaluates reconstruction, text-to-video generation, image-to-video generation, and RGB compatibility. On its RGBA reconstruction benchmark, UniRGBA reports a 4.16 dB PSNR improvement over a retrained Wan-Alpha baseline.

Critical strengths
  1. The method extends the causal VAE at its input and output boundaries lets the method keep the DiT’s 16-channel latent interface. This gives the design a coherent reason for avoiding a separate alpha-token stream.

  2. Selectively updating roughly 5% of the VAE parameters limits how much of the pretrained representation must be relearned.

Critical weaknesses
  1. RGBA representation learning, compositing losses, and latent constraints have precedents in AlphaVAE[1]; transparent-video generation already has TransPixeler[2] and Wan-Alpha[3]. The paper's contribution is narrowly defined as adapting a causal video VAE while retaining the pretrained DiT interface.

  2. Retaining the 16-channel latent width plausibly avoids alpha-token overhead, but the experiments do not establish the resulting differences in sequence length, memory, speed, or training cost under matched conditions. This leaves an important part of the method’s motivation supported by design reasoning rather than measurement.

[1] AlphaVAE: Unified End-to-End RGBA Image Reconstruction and Generation with Alpha-Aware Representation Learning [2] TransPixeler: Advancing Text-to-Video Generation with Transparency [3] Video Generation with Stable Transparency via Shiftable RGB-A Distribution Learner

Questions for authors
  1. At matched resolution and video length, how do token count, peak memory, inference time, and training cost compare with TransPixeler and Wan-Alpha?
  2. How much of the reconstruction gain comes from the architecture rather than additional training? Please compare the proposed channel expansion and selective unfreezing with alternative adaptations under the same data and update budget.
  3. Does Stage 2 help because of the GAN loss or because training continues?
  4. How representative is the internal RGBA benchmark? Please give a breakdown of transparent objects, diffuse effects, difficult boundaries, clip lengths, and source types, and specify which evaluation assets can be released.
Reviewer_dumZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes UniRGBA, an extention framework of pretrained RGB video diffusion model to support RGBA video generation. The main idea of the paper is a boundary adaption scheme on the Wan2.1 VAE while preserving the original 16-channel latent interface. Specifically, it fine-tunes only outer encoder/decoder layers, and introduces a KL reference loss to anchor the finetuned posterior to frozen RGB prior. The proposed method is evaluated over reconstruction, RGBA T2V/I2V, and RGB transfer.

Critical strengths
  1. The problem and design principle is practical and reasonable
    The core idea to keeping the pretrained DiT interface and concentrating modality adaptation in the VAE is established in related field (i.e, RGBA image generation) and well motivated.

  2. The evaluation is through and the emprical gain is substantial

Critical weaknesses
  1. The novelty appears to be mostly recipe-level
    The core idea of interface-preserving RGBA itself has close precedents in image generation, e.g, AlphaVAE. The main distinction of UniRGBA therefore seems to be extending that principle to video generation and a system-level recipe. However, most components in the design is not in itself novel. More critically, the video setting appears to introduce only practical level recipes without adding any new principle-level novelty to the proposed method.

  2. The boundary-only adaptation choice is not fully ablated
    The paper motivates boundary adaptation as a way to preserve the pretrained RGB core and limiting latent drift, which is a potentially interesting and distinctive idea. However, the current ablation seems not to have enough evidence to support this idea, as it only compares several boundary configurations, instead of comparing boundary adaption to, for example, more layers or even full adaptation with same interface.

Questions for authors
  1. Boundary adaptation Ablation: Present more ablation on the boundary adaptation choice, if that is reasonable.
  2. Novelty: Are there any video-specific challenges that fundamentally motivate the proposed boundary adaptation or other components, beyond engineering considerations, and could these be articulated more clearly at the principle level?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.