acceptodds
Withdrawn from ICLR 2027

AW-Composer: Compositional Asset and Weather Editing for Driving Videos

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Compositional editing of driving videos offers a scalable way to create long-tail scenarios by combining asset insertion with weather control. Given a background video, an asset with specified placement, and a retained or newly specified target weather, the goal is to preserve scene structure and asset identity while maintaining spatial and temporal weather consistency. These editing dimensions are coupled: weather transformation changes global appearance and visibility, whereas insertion requires local adaptation to the resulting scene. Achieving this requires coordinated global weather editing and local asset adaptation, yet content-aligned cross-weather supervision is difficult to obtain. We present AW-Composer, a diffusion-based framework that coordinates AW-Weather and AW-Harmony. AW-Weather transforms source videos to requested weather while preserving scene content and also provides paired self-insertion supervision. Trained on these pairs, AW-Harmony uses a single-step, mask-conditioned video DiT with a multi-scale Weather Context Adapter to adapt inserted assets across frames while preserving the scene outside the insertion region. We evaluate weather transformation, paired and open-set harmonization, and specified-weather joint composition. AW-Weather achieves a human score of 93.6/100, compared with 81.2 for VACE. On paired insertion, AW-Harmony reduces FVD by 42.8% relative to the best baseline and achieves the best Weather Consistency on both insertion tracks. On specified-weather joint composition, AW-Composer reduces FVD by 22.5% relative to the strongest harmonization baseline. In the six-way human study, AW-Harmony obtains a Top-1 rate of 49.2%, nearly three times that of the next-best method.

Primary area:
generative models
Keywords:
compositional video editing, weather editing, video harmonization, asset insertion, video diffusion models, autonomous driving

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YskoOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AW-Composer coordinates global weather transformation and local asset harmonization for driving videos. AW-Weather builds on WeatherWeaver and uses a Wan-based model with explicit weather conditioning and staged training on simulated, edited-image, and real-domain pseudo-pairs. It generates pseudo-clear counterparts of real adverse-weather videos; masked objects from these counterparts are pasted into the original videos to construct supervised training pairs for AW-Harmony, with the original real videos as targets. AW-Harmony uses separately trained Wan parameters, a single-step latent prediction, a multi-scale Weather Context Adapter, and a mask-aware reconstruction objective. The paper evaluates weather transformation, paired and open-set insertion, joint composition, round-trip recovery, and downstream perception.

Critical strengths
  1. Well-motivated task. Inserted assets require weather adaptation beyond ordinary color harmonization, and coordinating global weather control with local adaptation addresses a practical problem in driving-data augmentation.

  2. Sensible supervision and architecture. The self-insertion construction provides real adverse-weather targets from individual observations. The multi-scale context adapter and mask-aware objective are sensible responses to background weather cues and small-object supervision. Copying the background outside the mask also gives an explicit preservation mechanism.

  3. Complementary empirical evidence. Paired-track FVD improves from 26.4 for the strongest baseline to 15.1, while open-set human Top-1 preference reaches 49.2%. The nuScenes experiment uses equal fine-tuning budgets and includes weather-only augmentation, direct pasting, harmonized insertion, and joint augmentation; Rain AP improves from 22.23 to 24.95 for joint augmentation (Tables 3, 13, and 14).

Critical weaknesses
  1. Dependence on AW-Weather is insufficiently characterized AW-Harmony receives supervised pairs whose foreground inputs are generated by AW-Weather. If de-weathering changes object geometry or identity, these pairs contain structural discrepancies as well as weather discrepancies. Since the targets are original real observations, AW-Harmony may learn to repair generator-specific distortions alongside harmonization, potentially changing unfamiliar assets at inference. This is a plausible failure mechanism, not demonstrated inevitable error propagation. Appendix A.6 describes structural filtering for real-domain weather pairs and acknowledges imperfect correspondence, but residual object drift in harmonizer training inputs and sensitivity to generator quality are not quantified. The data contribution is therefore the construction of paired supervision, whose reliability still needs validation.

  2. The comparisons do not adequately isolate the methodological contribution. Shared test composites and masks ensure consistent inputs, but do not establish equal training opportunities. The baselines use official implementations, whereas AW-Harmony is trained with weather-specific paired supervision and a driving-adapted VAE. The color-only comparison establishes the value of the supervision, but does not show whether existing harmonizers trained on the same pairs would obtain similar gains. The context-pool and ControlNet ablations are useful, yet a plain mask-conditioned Wan baseline with the same data, VAE, and training budget is needed to identify the adapter’s incremental contribution.

  3. Independent validation of joint composition and identity preservation remains limited. The 150 joint-composition cases reinsert an original vehicle at its original location and use the complete AW-Weather output as the paired reference (Section 4.1; Appendix B.2). Agreement with that generated reference measures reconstruction within the proposed pipeline and cannot independently establish physical weather realism. Round-trip recovery also reuses backgrounds generated by AW-Weather, which limits independence. The open-set track and human study partly address this concern. However, Appendix B.5 acknowledges that the learned weather evaluator lacks calibration against human local-weather judgments, and the harmonization VLM prompt explicitly excludes geometric distortion and deformation (Figure 16). Consequently, these weather-consistency results do not fully substantiate preservation of novel asset geometry and identity.

  4. Novelty over closely related methods needs clearer isolation. AW-Weather explicitly builds on WeatherWeaver, which already introduced controllable weather synthesis/removal and a multi-source data strategy. Its quantitative weather comparisons do not include WeatherWeaver, and the contributions of the new conditioning paths are not isolated. Single-step latent prediction, low-rank adaptation, and context attention are individually established techniques. The coordinated application and weather-specific supervision may still be valuable contributions, but stronger evidence is needed to show what the proposed design adds beyond these ingredients.

  5. Reproducibility depends on substantial unreleased resources. The adapted VAE uses 100 hours of self-collected driving video and substantial training compute (Appendix A.2). The harmonization data and internal OCC/AEB evaluation also depend on non-public resources. Although many optimization details are provided, code and test-set release is conditional, and the internal deployment protocol remains confidential. Without the adapted VAE checkpoint and reproducible data-generation and filtering procedures, independent reproduction of the principal results remains difficult. The internal deployment results should therefore be treated as supporting case-study evidence.

Questions for authors
  1. Can you quantify object geometry/identity drift in the pseudo-clear foregrounds and report harmonizer sensitivity to AW-Weather quality, for example using Stage-1 versus Stage-2 outputs for the same source clips?

  2. Can you provide a harmonizer comparison using identical weather-specific pairs and comparable training budgets, together with a plain mask-conditioned Wan baseline using the same adapted VAE?

  3. Can you supplement the joint-composition results with an independent assessment of cross-source inserted assets, explicitly evaluating both local weather consistency and preservation of geometry/identity? How do the learned evaluator’s rankings relate to human judgments?

  4. What improvements are attributable to AW-Weather’s conditioning design beyond the WeatherWeaver-style curriculum? Can you provide a direct comparison or focused conditioning ablations?

  5. Will the release include the adapted VAE checkpoint, harmonizer weights, masks, data-generation scripts, and exact filtering settings needed to reproduce the main experiments?

2 comments

We thank the reviewer for the constructive feedback. We address each concern below.

Q1. Structural preservation of pseudo-clear foregrounds and sensitivity to upstream quality

To examine potential structural bias from generated inputs, we evaluate both training-pair quality and cross-source asset adaptation.

Preservation of training assets. Comparing temporally aligned original adverse-weather assets with pseudo-clear assets gives local DINOv2 of 0.7723 and CLIP-S of 0.8428. Asset-level VLM Preservation uses gpt-5.6-sol to assess identity, shape, color, position, and scale, ignoring weather appearance changes and judging only content visible in both crops. The normalized mean score is 91.8/100.

Effect of upstream filtering on downstream training. We separately train two AW-Weather Stage 2 models using filtered and unfiltered real-domain pairs. Each model then generates pseudo-clear assets to construct its own set of self-insertion pairs, and we train a separate AW-Harmony model on each set. With unfiltered upstream pairs, paired-track FVD increases from 15.1 to 19.7 and open-set Same-Weather decreases from 72.4% to 71.3%. This experiment examines how upstream pair filtering affects subsequent harmonization training outcomes.

Cross-source asset adaptation and preservation. Beyond training-pair checks, Track B inserts independently sourced assets into real adverse-weather backgrounds without using AW-Weather outputs as references. The added evaluation uses pre-harmonization assets as content references, with the same local crop expanded by 10% around the asset bounding box. AW-Harmony achieves local DINOv2 of 0.8244, CLIP-S of 0.8757, and Same-Weather of 72.4%.

These results support transfer of the adaptation ability learned through self-insertion supervision to cross-source assets and also evaluate content preservation in the final edited outputs. We report training-pair preservation separately from cross-source output preservation; the full method comparison is in Q3.

Q2. Harmonization baselines under matched training conditions

Table 7 already includes a mask-conditioned Wan baseline without the Weather Context Adapter, using standard ControlNet-style background conditioning. The comparison keeps the training pairs, adapted VAE, training budget, loss, and backbone unchanged. WC is 0.492 with the full Adapter versus 0.465 for the control; Same-Weather is 72.4% versus 68.4%. This replacement ablation directly examines the gains from the specialized context-adaptation design over general conditioning injection.

We also added same-pair training of existing models. Both DiffusionHarmonizer and VACE improve after training on the same pairs, while AW-Harmony still achieves lower FVD and higher open-set weather consistency. DiffusionHarmonizer and AW-Harmony both use 30,000 updates. Fine-tuning reduces DiffusionHarmonizer's FVD from 29.9 to 19.3 and increases open-set Same-Weather from 58.9% to 66.9%. VACE, using the same Wan backbone and adapted VAE, achieves FVD of 18.3 and open-set Same-Weather of 69.9% after fine-tuning on the same pairs, compared with 15.1 and 72.4% for AW-Harmony.

Full results and training settings are provided in C2.

Q3. Independent cross-source assets, content preservation, and evaluator validation

We first clarify that the paper's open-set asset-insertion track already includes independent cross-source evaluation: assets from independent sources are inserted into real adverse-weather backgrounds, and evaluation does not use AW-Weather-generated results as reference targets.

Asset content preservation. Using the inserted assets before harmonization as references, we apply the same local crop expanded by 10% around the asset bounding box. Scores are averaged within each sequence and then across sequences.

MethodLocal DINOv2 ↑Local CLIP-S ↑Same-W. (%) ↑
VTT0.66850.813967.8
Harmonizer0.79770.871660.7
LBM0.87960.907965.4
VACE-1.3B (fine-tuned)0.81040.872369.9
DiffusionHarmonizer (public release)0.80260.873858.9
AW-Harmony0.82440.875772.4

DINOv2 and CLIP-S measure content preservation, not weather-harmonization quality. Although AW-Harmony trails LBM on content-preservation scores, it achieves better weather consistency: Same-Weather is 72.4%, compared with 65.4% for LBM.

Weather evaluator and human judgments. We conducted a blind evaluation using 50 clips in rain, fog, and snow and six methods, uniformly sampling 300 within-sequence method pairs. The evaluator and its scoring parameters were fixed before collecting human judgments, and method names were hidden. Annotators selected the result with better local weather consistency, and we compared their choices with the evaluator's pairwise winners.

WeatherEvaluator–human agreement on the pairwise winner ↑
Rain78.00%
Fog83.00%
Snow75.00%

Q4. Independent gains from AW-Weather conditioning

With the same Wan backbone, adding an independent Weather AdaLN path improves VLM-Weather from 88.5 to 91.1, VLM-Technical from 79.7 to 82.8, and CLIP-S from 0.751 to 0.763. This comparison evaluates weather tokens alone against weather tokens with Weather AdaLN; full results are in Common Response C1.

With weather tokens alone, weather conditions and text share cross-attention. We observed incomplete weather removal or insufficient responses to the target intensity. Weather AdaLN injects source intensity, target intensity, and their difference through an independent path, improving these behaviors.

We separately ablate real-domain identity-mapping supervision using the same 15,000 training updates: aesthetic quality increases from 45.31 to 46.92, and imaging quality from 66.29 to 67.20. We will add the conditioning-path and real-domain training comparisons separately in the revision.

W5 / Q5. Reproducibility

Planned resources include the adapted VAE checkpoint, AW-Weather and AW-Harmony weights, releasable data and corresponding masks, data-generation and filtering scripts, and training, inference, and evaluation configurations. These resources are scheduled for release after paper acceptance and completion of the company's internal review.

We will also provide the filtering thresholds actually used, the filtering order and parameter settings, scene-level splits, and sample indices. The revised reproducibility statement will list the resources and release arrangements.

Among the public driving-video datasets we examined, adverse-weather samples were predominantly rainy. We did not find sufficient snow and fog videos meeting the task requirements, so we selected the corresponding videos from an internal database. The scope of releasable data will be specified with the resource list.

Reviewer_5BXeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AW-Composer, a diffusion-based framework for compositional asset and weather editing of driving videos. The task is to insert a specified asset into a background video while preserving scene structure and asset identity/placement/motion, and making the asset consistent with either the retained or a newly specified weather condition. The framework decomposes this coupled problem into two cooperating modules built on Wan2.1-T2V-1.3B DiTs with a shared driving-adapted VAE: (1) AW-Weather performs global weather transformation with continuous 7-dimensional weather-intensity control, adapted from WeatherWeaver's source-conditioned editing and staged training; it additionally produces pseudo-clear counterparts of real adverse-weather videos to construct "self-insertion" paired supervision (clean asset pasted back into its original position in the real adverse video, with the real observation as target). (2) AW-Harmony performs single-step, mask-guided conditional transport from the composite latent, in which a multi-scale Weather Context Adapter lets object tokens actively read background weather evidence from near-object, lower-frame, and global pools, and a mask-aware loss reweights object/background/boundary regions. A learned ResNet-50 Weather Consistency evaluator is introduced for the no-reference open-set setting. The paper establishes a Paired Self-Insertion track, an Open-Set Asset Insertion track, a specified-weather joint composition track, and a harmonization-first round-trip protocol. Experiments show consistent improvements over strong baselines (VACE, LBM, DiffusionHarmonizer, etc.): AW-Weather attains a human score of 93.6 vs 81.2 for VACE; AW-Harmony reduces FVD by 42.8% on the paired track and the full system reduces FVD by 22.5% on joint composition; a six-way human study gives 49.2% top-1 share. Downstream nuScenes fine-tuning and an internal OCC/AEB deployment further support the data-utility claim.

Critical strengths

(I) A well-motivated problem formulation with a clever supervision strategy. The paper correctly identifies that weather transformation (global) and asset adaptation (local) are coupled editing dimensions whose combination matters for synthetic-data utility, and that content-aligned cross-weather pairs are essentially uncollectable. The self-insertion construction — reinserting the weather model's pseudo-clear asset into the original adverse observation — converts an uncollectable data problem into a scalable self-supervised one. The supervision ablation (WC 0.427→0.492, Same-Weather 57.5%→72.4% over color-only training, Table 7/11) directly validates that this pairing, not generic appearance augmentation, is what drives the gains. This is consequential because it establishes a reusable recipe for any local-consistency editing task lacking paired data. (II) Comprehensive and unusually honest evaluation. The paper builds complementary tracks (paired / open-set / joint composition / round-trip), introduces a learned evaluator with explicit scene-disjointness and calibration disclosures, adds human and VLM studies with confidence intervals, and closes the loop with downstream detection and deployment evidence. The fact that single-step AW-Harmony improves FVD by 42.8% over the best multi-step baseline while maintaining full-frame quality (Table 3/4) — and that the paper honestly flags where its evaluator is descriptive rather than validated — increases my confidence that the reported gains are real rather than metric artifacts.

Critical weaknesses

(I) Limited methodological novelty at the component level. The paper's contributions lie predominantly in task formulation and data construction rather than in learning algorithms. AW-Weather is explicitly built on WeatherWeaver's architecture, data strategy, and staged curriculum (the paper says so in Sec. 2/3.2); AW-Harmony's single-step prediction at a fixed transport coordinate is a member of the bridge/flow one-step translation family (LBM); and the Weather Context Adapter is a well-executed combination of known ingredients (regional cross-attention, learned gating, zero-initialized residual injection). The ablations show each choice helps, but none of the individual mechanisms is new. For ICLR, where methodological originality is a central criterion, this limits the paper's contribution to the formulation-and-benchmark layer, and I would like the authors to be more explicit about which parts are inherited versus new. (II) Circularity and generalization boundaries. Both the training pairs and the joint-composition targets derive from AW-Weather outputs, so the framework's headline numbers are partially a measure of its own weather model rather than of the harmonizer in isolation; the strongest deployment claim (AEB recall +5.5pp) is unverifiable. Meanwhile, generalization is tested on a narrow slice: 21-frame clips at 832×480, and three weather types in the open-set track. The paper's qualitative examples are vehicle-centric (Figs. 6, 12, 13), yet the asset composition of the open-set evaluation set is not reported (Appendix B.1 gives only the weather distribution), so the claimed object diversity of 23 training categories is never verified at evaluation time. In training data, dust has only 7 of 26,600 clips. No evidence is given for mixed weather, non-vehicle assets, longer sequences beyond the sliding-window deployment note, or sensitivity to mask quality. These boundaries leave open the question of whether the gains transfer beyond the studied regime. (III) Insufficient quantitative comparison with the nearest prior work. Two conspicuous omissions weaken the evaluation's claim of state-of-the-art. (1) WeatherWeaver (ICCV 2025): AW-Weather is explicitly built on WeatherWeaver's source-conditioned editing and staged training (Sec. 2, Sec. 3.2), making it the single most relevant weather-editing baseline; yet the weather track compares only FRESCO, Cosmos Transfer2.5-2B, and VACE-1.3B — generic editors none of which were designed for controllable weather synthesis — while WeatherWeaver appears only as a checkmark row in Table 1 with no quantitative numbers. Consequently, readers cannot assess how much of AW-Weather's 93.6 human score derives from the inherited WeatherWeaver recipe versus the paper's own modifications (7D target control, driving-adapted VAE, pseudo-pair curriculum). Since WeatherWeaver is public and open-source, this comparison is feasible and its absence is not excusable by availability. (2) Mirage (arXiv 2512.24227, Dec 2025): cited in the paper's Introduction and Related Work via the "Wang et al., 2025" reference group (and listed by name in the bibliography), Mirage is a one-step video diffusion model for driving-scene asset editing with alignment-aware supervision — the closest published match to AW-Harmony's technical profile apart from the weather dimension. Yet it is absent from all five harmonization baselines. Given that Mirage's code is public, a direct comparison (at least on the paired self-insertion track, to which Mirage can be adapted) is needed to substantiate the claim that AW-Harmony outperforms existing one-step driving asset editors. These two omissions together make the "outperforms state-of-the-art" claims in Sec. 4.2/4.3 over-broad relative to what was actually measured.

Questions for authors

(I). What fraction of self-insertion training pairs survives the structural filtering in A.6, and is there any quantitative estimate of residual identity/geometry misalignment in the surviving pairs? Does AW-Harmony's performance degrade when trained on unfiltered pairs? (II). For the six-way human study, what were the raters' backgrounds and the inter-rater agreement? Was the study preregistered or whose design choices (e.g., 30-31 clips per rater) were fixed in advance? (III). The open-set track covers heavy rain, fog, and heavy snow, and the paper's qualitative examples are vehicle-centric (Figs. 6, 12, 13), yet the asset composition of the evaluation set is not reported. How does AW-Harmony behave on non-vehicle assets (pedestrians, irregular obstacles) and mixed/hybrid weather? (IV). Why are WeatherWeaver and Mirage absent from the quantitative comparisons? (a) AW-Weather is explicitly built on WeatherWeaver (Sec. 3.2), and WeatherWeaver's code is public — a direct comparison on your 200-video clear-target protocol would isolate the contribution of your modifications (7D control, driving VAE, pseudo-pair curriculum). (b) Mirage (cited via "Wang et al., 2025" in your Introduction and Related Work, listed by name in the bibliography) shares AW-Harmony's core profile — one-step video diffusion for driving asset editing, public code; adapting it to the paired self-insertion track seems feasible. Please either add these comparisons or explain why they are not applicable.

3 comments

We thank the reviewer for the constructive feedback. We address each concern below.

W1. Methodological contributions and relation to prior techniques

Thank you for suggesting that we clarify the contribution. For adverse-weather asset harmonization in driving videos, our methodological contributions center on self-insertion supervision construction and background-weather-driven local adaptation. Self-insertion generates pseudo-clear foregrounds from real adverse-weather videos and pastes them back into the original backgrounds, using the original real observations as targets. This provides a way to construct supervision for cross-weather asset harmonization without directly collecting paired observations across weather conditions.

In implementation, AW-Weather draws on WeatherWeaver for source-video-conditioned editing, intensity representation, and training with multiple data sources, while building the model on Wan. Comparisons of its conditioning path and real-domain training are provided in C1. AW-Harmony lets asset tokens read weather evidence from near-object, lower-frame, and global background regions, and uses a mask-aware objective to strengthen supervision on assets and their boundaries.

We examine these designs through controlled experiments. With the same backbone, training pairs, adapted VAE, training budget, and loss, the Weather Context Adapter improves WC from 0.465 to 0.492 and Same-Weather from 68.4% to 72.4% over general ControlNet-style background injection (Table 7). Multiscale-context and loss ablations further examine the local adaptation design; fine-tuning DiffusionHarmonizer and VACE on the same pairs validates the value of self-insertion supervision for existing models (C2).

Q1. Pair filtering, asset preservation, and the effect of filtering

Appendix A.6 filters real-domain training pairs for AW-Weather Stage 2. Counting 21-frame video pairs, 42,052 of the 53,758 candidates are retained (78.22%) and 11,706 are rejected (21.78%). The retained set has global DINOv2 and CLIP-S similarities of 0.7840 and 0.8646, respectively, compared with 0.7598 and 0.8466 for the rejected set.

Asset preservation. Comparing temporally aligned original adverse-weather and pseudo-clear assets gives local DINOv2 of 0.7723 and CLIP-S of 0.8428. Asset-level VLM Preservation uses gpt-5.6-sol to score identity, shape, color, position, and scale from 0 to 4, ignoring weather appearance changes and judging only content visible in both crops. Insufficient visibility is recorded as null. The normalized mean score is 91.8/100.

Effect of unfiltered pairs. When AW-Weather Stage 2 is trained with unfiltered pairs, both reconstruction quality and weather consistency in downstream harmonization decrease.

Evaluation track and metricFiltered pairsUnfiltered pairs
Paired self-insertion: FID ↓6.57.5
Paired self-insertion: FVD ↓15.119.7
Paired self-insertion: CLIP-GT ↑0.9670.961
Open-set insertion: Same-W. (%) ↑72.471.3
Open-set insertion: Embedding Distance ↓0.2580.264

The preservation scores and filtering comparison examine training-pair reliability; cross-source asset adaptation evaluates whether the ability learned from this supervision transfers to other inputs (Q3).

Q2. Human-study design and agreement

Twenty-five members of a dedicated data-annotation team conducted blinded six-way Top-1 comparisons on a custom platform, evaluating only local weather consistency. The study was not formally preregistered, but the task, clips, judgments per clip, candidate-position balancing, and task allocation were fixed before voting. Clips were manually screened for target visibility in the original inputs without blinding; method names were hidden during voting.

Each of the 152 clips was judged by five different participants, yielding 760 choices, with each person judging 30 or 31 clips. We retained the original choices and report agreement below.

Agreement on the preferred methodClipsProportion
At least 3 of 5 annotators agreed101/15266.45%
At least 4 of 5 annotators agreed77/15250.66%
All 5 annotators agreed34/15222.37%

W(II) / Q(III). Evaluation independence, asset coverage, and generalization

Evaluation independence. Paired self-insertion uses original real adverse-weather videos as targets; AW-Weather only generates the pseudo-clear foreground inputs. Table 3 therefore evaluates reconstruction of real observations. The joint-composition track uses AW-Weather outputs as references.

Independent validation also includes open-set human evaluation and detection on unedited real nuScenes images. All groups use the same detector initialization and fine-tuning budget. Harmonization improves Rain AP by 0.93 points over direct pasting, and joint augmentation improves it by 2.72 points over training with real data alone (Table 13). The internal AEB improvement of 5.5 percentage points is presented as a supplementary deployment case.

Asset coverage. We audited asset configurations for 475 open-set videos. Sources include water-filled barriers, boxes, cones, bollards, crash barriers, traffic signs, warning devices, trash bags, handcarts, stools, and bicycle/e-bike models. Non-vehicle assets are already included: the inserted objects in Figure 6 are water-filled barriers, while the pedestrian in Figure 12 belongs to the paired track. We will add the open-set asset inventory and report it separately from the 23 training asset categories.

Generalization scope. Current quantitative harmonization evaluation uses 21-frame videos at 832×480, with Track B covering three dominant weather conditions: heavy rain, fog, and heavy snow. There are only seven real dust-weather training clips, and no independent dust test set has been established. Mixed weather, longer sequences, and mask perturbations remain to be evaluated quantitatively. Sliding-window processing for long-video deployment will be described separately in the revision.

Q4. Related baselines

We added a WeatherWeaver-style removal comparison with a common Wan backbone. With the same 200 videos, clear-weather target, and 30,000 training updates, AW-Weather Stage 1 achieves VLM-Weather, VLM-Technical, and CLIP-S of 91.1, 82.8, and 0.763, compared with 87.4, 78.5, and 0.744 for WeatherWeaver-style. Common Response C1 provides the full settings and ablations of conditioning and real-domain training.

Mirage is closely related work on single-step video asset editing, but its official repository still lists model weights as forthcoming. We use the publicly available single-step video harmonization method DiffusionHarmonizer and fine-tune it for 30,000 updates on the same pairs as AW-Harmony. Its paired FVD decreases from 29.9 to 19.3, compared with 15.1 for AW-Harmony; full results are in C2. If Mirage releases its model weights, we will add a direct comparison.

Thank you for the response. The same-pair fine-tuning results and ablations are appreciated, but my main concerns remain: the evaluation pipeline is largely internal and self-consistent, and generalization beyond three weather conditions on short clips is untested. I maintain my rating

Reviewer_bWhHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents AW-Composer, which unifies asset insertion and weather editing for driving videos. To address the challenges of obtaining paired training data across different weather conditions and the difficulty of maintaining consistency between inserted objects and backgrounds, the authors introduce two key components. AW-Weather enables controllable global weather translation and constructs paired self-supervised data for object insertion. AW-Harmony adopts a mask-conditioned video DiT model and introduces a weather context adapter to align inserted objects with surrounding weather conditions. Experimental results demonstrate promising performance in weather editing quality and consistency. Human evaluation and downstream detection tasks further validate the effectiveness.

Critical strengths

The task formulation carries practical value. Autonomous driving perception demands long-tail samples under adverse weather, which incur high costs for real-world collection. This work investigates weather consistency between inserted objects and backgrounds, which is essential for generating realistic long-tail driving scenarios. Furthermore, the authors devise a self-insertion strategy to construct paired supervision data, mitigating the scarcity of real paired training data.

Critical weaknesses

The methodological novelty is limited. The proposed AW-Weather is largely built upon the existing WeatherWeaver framework, with major modifications including a DiT backbone, a dual-path weather conditioning injection mechanism, and revised training schemes. Similarly, AW-Harmony integrates established techniques such as multi-scale cross-attention and gated feature fusion. Although the combination of these components yields practical utility, the paper does not sufficiently ablate the necessity and individual contribution of each design. It remains unclear whether performance gains stem from the proposed technical innovations, or from a stronger pre-trained backbone and additional training data.

Questions for authors
  1. What are the key differences between AW-Weather and WeatherWeaver? Could the authors provide ablation studies?
  2. How does the proposed framework compare against the simple baseline that first inserts the target asset and then applies global weather transformation?
  3. How are new objects inserted into target scenes? Specifically, how are their geometry, motion trajectories, and occlusion relationships determined?
  4. The quantitative comparisons are mainly conducted against general video editing methods, while AW-Composer is trained on a custom dataset. Are the reported performance improvements attributed to the proposed network architecture, or to task-specific training?
  5. Table 1 lists several specialized weather editing methods for driving scenarios, yet these methods are excluded from quantitative comparisons. Could the authors provide comparisons with them?
2 comments

We thank the reviewer for the constructive feedback. We address each concern below.

W1 / Q1. Differences from WeatherWeaver

AW-Weather is built on Wan as a unified weather-editing model that directly converts source weather to target weather. It uses an independent Weather AdaLN path and real-domain identity-mapping supervision to improve intensity control and real-domain adaptation, respectively. WeatherWeaver uses separately trained weather-removal and synthesis models.

With a common Wan backbone, VLM-Weather is 91.1 for AW-Weather and 87.4 for WeatherWeaver-style. In the conditioning ablation, adding Weather AdaLN improves this score from 88.5 to 91.1. With the same 15,000 updates, identity mapping improves aesthetic quality from 45.31 to 46.92 and imaging quality from 66.29 to 67.20. Full comparisons are provided in C1.

Q2. Inserting assets before global weather editing

Generating the target-weather background first, then inserting and locally harmonizing assets, gives slightly better detection results than inserting assets before global weather editing. We added the baseline of inserting assets into clear-weather scenes and then converting the entire scene to rain, and compared it with WA: generating the rainy background first, then inserting assets and applying local harmonization. Both groups use the same scenes, assets, placements, masks, and target intensities, as well as the same detector initialization, fine-tuning updates, and 1:1 real/generated training-data ratio. We test on unedited real nuScenes validation images and report AP@[0.50:0.95].

Training-data generation procedureClear APRain AP
Insert assets, then globally edit weather27.0824.43
WA: edit weather, then insert and harmonize27.2324.95

WA improves Clear AP and Rain AP by 0.15 and 0.52 points, respectively. One possible reason is that subsequent global weather generation weakens the contours, textures, and local contrast of small objects, removing recognition cues. WA completes local adaptation after global conversion, helping preserve this information.

Inserting and harmonizing assets directly in real adverse-weather videos also retains the real background outside the mask, reducing the bias introduced by background regeneration. In Table 13, harmonization improves Rain AP by 0.93 points and Clear AP by 0.41 points over direct pasting, demonstrating the value of local adaptation for perception training.

Q3. Asset insertion procedure

The object insertion algorithm is largely consistent with prior work; see SynthDrive: Scalable Real2Sim2Real Sensor Simulation Pipeline for High-Fidelity Asset Generation and Driving Data Synthesis.

Object geometry and textures are generated using an image-to-3D algorithm, followed by manual quality checks and adjustments. The objects are then placed at predefined locations and rendered using Gaussian splatting (GS) or the Blender API.

Motion trajectories are constrained by the trajectories and 3D bounding boxes of moving objects in the real scene, including the ego vehicle, to prevent collisions.

Occlusions are determined using 3D bounding boxes for dynamic objects and metric depth for static objects.

W1 / Q4. Task-specific supervision and architecture

Both self-insertion supervision and the local adaptation design are supported by controlled comparisons. Fine-tuning on the same pairs reduces FVD from 29.9 to 19.3 for DiffusionHarmonizer and from 31.9 to 18.3 for VACE, showing that the supervision we construct also benefits existing models.

The gains from local design are examined through a same-backbone replacement ablation: replacing the Weather Context Adapter with general ControlNet-style background injection, while keeping the training pairs, adapted VAE, budget, and loss unchanged, decreases WC from 0.492 to 0.465 and Same-Weather from 72.4% to 68.4%. Performance therefore also depends on how background weather information is read and injected.

Under the same paired supervision, the complete AW-Harmony achieves FVD of 15.1 and open-set Same-Weather of 72.4%, outperforming both fine-tuned baselines. Multiscale-context and loss ablations are in Tables 7/10/12, and the same-data training results are in C2.

Q5. Specialized weather-editing baselines

Our weather-editing evaluation takes adverse-weather RGB videos as inputs and clear weather as the target, testing the weather-removal capability required to construct self-insertion pairs.

WeatherEdit's public pipeline primarily targets weather synthesis and does not provide a removal mode applicable to this protocol. AutoWeather4D requires preprocessed scene H5 inputs, but its official repository still lists the corresponding preprocessing tools and instructions as TODO. Its public release therefore lacks the preprocessing pipeline needed to construct aligned inputs from our RGB test clips. We consequently compare video-editing models that can directly process RGB videos and perform weather removal, using the same test clips, target weather, and evaluation metrics.

Comments on the paper

2 comments

We thank the three reviewers for recognizing the practical importance of adverse-weather asset harmonization and the potential of self-insertion supervision for other tasks with scarce paired data. Our core contribution is self-insertion supervision: we generate pseudo-clear assets from real adverse-weather videos and paste them back into their original backgrounds, using the original real videos as harmonization targets. This constructs cross-weather training pairs that are difficult to collect directly. Around this supervision strategy, AW-Composer combines global weather editing with local asset adaptation and validates its practical effectiveness through cross-source assets, human evaluation, and real downstream tasks. We first address the shared concerns below.

C1. AW-Weather and WeatherWeaver

AW-Weather learns a unified conversion conditioned on source and target weather using Wan, providing pseudo-clear foregrounds for self-insertion supervision. We draw on WeatherWeaver for source-video-conditioned editing, intensity representation, and training with multiple data sources. The two methods differ in conversion modeling, weather-conditioning injection, and real-domain training, as examined below.

1. Weather removal with the same backbone. WeatherWeaver uses two SVD models with identical architectures but independent parameters for weather removal and synthesis, respectively. AW-Weather uses a single Wan model to perform the conversion according to source and target weather conditions.

Since we could not find usable official WeatherWeaver checkpoints, we separately constructed a WeatherWeaver-style (Wan) baseline with the same Wan2.1 pretrained weights and VAE as AW-Weather. It retains separately trained removal and synthesis models, the clear-weather intermediate domain, and channel-wise intensity-map concatenation, using the same seven-dimensional continuous weather labels. The baseline uses the simulation videos and Qwen-edited image pairs shared by both methods; AW-Weather additionally uses real-domain identity-mapping supervision and the corresponding real data.

The baseline removal model and AW-Weather Stage 1 are both trained for 30,000 updates and evaluated on the same 200 adverse-weather videos with clear weather as the target.

ConfigurationVLM-Weather ↑VLM-Technical ↑CLIP-S ↑
AW-Weather (Stage 1)91.182.80.763
WeatherWeaver-style (Wan, Stage 1)87.478.50.744

2. Conditioning comparisons within AW-Weather. With weather tokens alone, weather and text share cross-attention, and we observed incomplete weather removal or insufficient responses to the target intensity. AW-Weather therefore uses an independent Weather AdaLN path to modulate network features with source intensity, target intensity, and their difference.

We compare the following three configurations within the same model. Intensity-map concatenation follows WeatherWeaver's conditioning approach.

AW-Weather configurationVLM-Weather ↑VLM-Technical ↑CLIP-S ↑
WeatherWeaver-style intensity-map concatenation88.279.90.750
Weather tokens only88.579.70.751
Weather tokens + Weather AdaLN91.182.80.763

3. Real-domain identity mapping. To reduce the simulation-to-real gap in AW-Weather Stage 1, we add identity-mapping supervision using real videos. With the same 15,000 updates, VBench aesthetic quality on real-domain weather-conversion outputs increases from 45.31 to 46.92, and imaging quality from 66.29 to 67.20, both on a 0–100 scale.

C2. Task-specific training and harmonization gains

Self-insertion supervision improves existing models, while AW-Harmony's local adaptation design provides further gains under the same supervision. We fine-tune DiffusionHarmonizer and VACE on the same pairs to evaluate the benefits of the supervision. Both baselines use the same training data as AW-Harmony and are trained for 30,000 updates. VACE additionally shares the Wan backbone and adapted VAE checkpoint with AW-Harmony.

Paired track (Track A).

MethodPSNR ↑SSIM ↑LPIPS ↓FID ↓FVD ↓WC ↑Emb. Dist. ↓Same-W. (%) ↑CLIP-GT ↑
DiffusionHarmonizer (public release)31.00.9750.0337.729.90.4920.26266.60.957
DiffusionHarmonizer (fine-tuned, 30k)33.70.9810.0286.719.30.5260.25173.70.965
VACE (official weights)31.40.9640.04610.631.90.5160.24672.00.939
VACE (fine-tuned)33.20.9670.0306.918.30.5250.23373.80.959
AW-Harmony33.50.9660.0286.515.10.5460.22875.30.967

Open-set track (Track B).

MethodWC ↑Emb. Dist. ↓Same-W. (%) ↑Int. Dist. ↓VBench ↑
DiffusionHarmonizer (public release)0.4320.30158.90.2990.778
DiffusionHarmonizer (fine-tuned, 30k)0.4760.28166.90.2820.776
VACE (official weights)0.4730.27066.90.2810.767
VACE (fine-tuned)0.4830.26769.90.2770.775
AW-Harmony0.4920.25872.40.2610.775

Transferability of the supervision. Fine-tuning on the same pairs reduces FVD from 29.9 to 19.3 for DiffusionHarmonizer and from 31.9 to 18.3 for VACE, showing that self-insertion supervision also benefits existing models. Tables 7/11 additionally compare weather-specific paired supervision with color-only supervision under a fixed harmonization architecture to examine the value of these pairs.

Gains from the Adapter design. The ControlNet variant in Table 7 removes the Weather Context Adapter and uses general background-conditioning injection, keeping the backbone, training pairs, adapted VAE, budget, and loss unchanged. The full Adapter improves WC from 0.465 to 0.492 and Same-Weather from 68.4% to 72.4%. Multiscale-context and loss ablations further examine the individual designs.

The complete AW-Harmony achieves FVD of 15.1 and open-set Same-Weather of 72.4%, outperforming both DiffusionHarmonizer and VACE fine-tuned on the same pairs. These comparisons examine the benefits of supervision construction, Adapter design, and the complete method, respectively.

Fine-tuning settings. DiffusionHarmonizer is initialized from the official temporal checkpoint, retaining its native Cosmos DiT, VAE, and single-step prediction mechanism. We update the DiT and VAE encoder/decoder, keeping time_conv frozen, and use AdamW with a learning rate of 2e-5.

For VACE, we fully update the condition-input mapping, control branches, patch_embedding, time_projection, and modulation. The self-attention and cross-attention Q/K/V projections in the main DiT use LoRA (rank=alpha=96). The VAE, T5, and remaining backbone parameters are frozen. The peak learning rate is 2e-5 for patch_embedding and 1e-4 for all other trainable parameters.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.