- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
AW-Composer coordinates global weather transformation and local asset harmonization for driving videos. AW-Weather builds on WeatherWeaver and uses a Wan-based model with explicit weather conditioning and staged training on simulated, edited-image, and real-domain pseudo-pairs. It generates pseudo-clear counterparts of real adverse-weather videos; masked objects from these counterparts are pasted into the original videos to construct supervised training pairs for AW-Harmony, with the original real videos as targets. AW-Harmony uses separately trained Wan parameters, a single-step latent prediction, a multi-scale Weather Context Adapter, and a mask-aware reconstruction objective. The paper evaluates weather transformation, paired and open-set insertion, joint composition, round-trip recovery, and downstream perception.
Critical strengths
-
Well-motivated task. Inserted assets require weather adaptation beyond ordinary color harmonization, and coordinating global weather control with local adaptation addresses a practical problem in driving-data augmentation.
-
Sensible supervision and architecture. The self-insertion construction provides real adverse-weather targets from individual observations. The multi-scale context adapter and mask-aware objective are sensible responses to background weather cues and small-object supervision. Copying the background outside the mask also gives an explicit preservation mechanism.
-
Complementary empirical evidence. Paired-track FVD improves from 26.4 for the strongest baseline to 15.1, while open-set human Top-1 preference reaches 49.2%. The nuScenes experiment uses equal fine-tuning budgets and includes weather-only augmentation, direct pasting, harmonized insertion, and joint augmentation; Rain AP improves from 22.23 to 24.95 for joint augmentation (Tables 3, 13, and 14).
Critical weaknesses
-
Dependence on AW-Weather is insufficiently characterized AW-Harmony receives supervised pairs whose foreground inputs are generated by AW-Weather. If de-weathering changes object geometry or identity, these pairs contain structural discrepancies as well as weather discrepancies. Since the targets are original real observations, AW-Harmony may learn to repair generator-specific distortions alongside harmonization, potentially changing unfamiliar assets at inference. This is a plausible failure mechanism, not demonstrated inevitable error propagation. Appendix A.6 describes structural filtering for real-domain weather pairs and acknowledges imperfect correspondence, but residual object drift in harmonizer training inputs and sensitivity to generator quality are not quantified. The data contribution is therefore the construction of paired supervision, whose reliability still needs validation.
-
The comparisons do not adequately isolate the methodological contribution. Shared test composites and masks ensure consistent inputs, but do not establish equal training opportunities. The baselines use official implementations, whereas AW-Harmony is trained with weather-specific paired supervision and a driving-adapted VAE. The color-only comparison establishes the value of the supervision, but does not show whether existing harmonizers trained on the same pairs would obtain similar gains. The context-pool and ControlNet ablations are useful, yet a plain mask-conditioned Wan baseline with the same data, VAE, and training budget is needed to identify the adapter’s incremental contribution.
-
Independent validation of joint composition and identity preservation remains limited. The 150 joint-composition cases reinsert an original vehicle at its original location and use the complete AW-Weather output as the paired reference (Section 4.1; Appendix B.2). Agreement with that generated reference measures reconstruction within the proposed pipeline and cannot independently establish physical weather realism. Round-trip recovery also reuses backgrounds generated by AW-Weather, which limits independence. The open-set track and human study partly address this concern. However, Appendix B.5 acknowledges that the learned weather evaluator lacks calibration against human local-weather judgments, and the harmonization VLM prompt explicitly excludes geometric distortion and deformation (Figure 16). Consequently, these weather-consistency results do not fully substantiate preservation of novel asset geometry and identity.
-
Novelty over closely related methods needs clearer isolation. AW-Weather explicitly builds on WeatherWeaver, which already introduced controllable weather synthesis/removal and a multi-source data strategy. Its quantitative weather comparisons do not include WeatherWeaver, and the contributions of the new conditioning paths are not isolated. Single-step latent prediction, low-rank adaptation, and context attention are individually established techniques. The coordinated application and weather-specific supervision may still be valuable contributions, but stronger evidence is needed to show what the proposed design adds beyond these ingredients.
-
Reproducibility depends on substantial unreleased resources. The adapted VAE uses 100 hours of self-collected driving video and substantial training compute (Appendix A.2). The harmonization data and internal OCC/AEB evaluation also depend on non-public resources. Although many optimization details are provided, code and test-set release is conditional, and the internal deployment protocol remains confidential. Without the adapted VAE checkpoint and reproducible data-generation and filtering procedures, independent reproduction of the principal results remains difficult. The internal deployment results should therefore be treated as supporting case-study evidence.
Questions for authors
-
Can you quantify object geometry/identity drift in the pseudo-clear foregrounds and report harmonizer sensitivity to AW-Weather quality, for example using Stage-1 versus Stage-2 outputs for the same source clips?
-
Can you provide a harmonizer comparison using identical weather-specific pairs and comparable training budgets, together with a plain mask-conditioned Wan baseline using the same adapted VAE?
-
Can you supplement the joint-composition results with an independent assessment of cross-source inserted assets, explicitly evaluating both local weather consistency and preservation of geometry/identity? How do the learned evaluator’s rankings relate to human judgments?
-
What improvements are attributable to AW-Weather’s conditioning design beyond the WeatherWeaver-style curriculum? Can you provide a direct comparison or focused conditioning ablations?
-
Will the release include the adapted VAE checkpoint, harmonizer weights, masks, data-generation scripts, and exact filtering settings needed to reproduce the main experiments?
2 comments
Q3. Independent cross-source assets, content preservation, and evaluator validation
We first clarify that the paper's open-set asset-insertion track already includes independent cross-source evaluation: assets from independent sources are inserted into real adverse-weather backgrounds, and evaluation does not use AW-Weather-generated results as reference targets.
Asset content preservation. Using the inserted assets before harmonization as references, we apply the same local crop expanded by 10% around the asset bounding box. Scores are averaged within each sequence and then across sequences.
| Method | Local DINOv2 ↑ | Local CLIP-S ↑ | Same-W. (%) ↑ |
|---|---|---|---|
| VTT | 0.6685 | 0.8139 | 67.8 |
| Harmonizer | 0.7977 | 0.8716 | 60.7 |
| LBM | 0.8796 | 0.9079 | 65.4 |
| VACE-1.3B (fine-tuned) | 0.8104 | 0.8723 | 69.9 |
| DiffusionHarmonizer (public release) | 0.8026 | 0.8738 | 58.9 |
| AW-Harmony | 0.8244 | 0.8757 | 72.4 |
DINOv2 and CLIP-S measure content preservation, not weather-harmonization quality. Although AW-Harmony trails LBM on content-preservation scores, it achieves better weather consistency: Same-Weather is 72.4%, compared with 65.4% for LBM.
Weather evaluator and human judgments. We conducted a blind evaluation using 50 clips in rain, fog, and snow and six methods, uniformly sampling 300 within-sequence method pairs. The evaluator and its scoring parameters were fixed before collecting human judgments, and method names were hidden. Annotators selected the result with better local weather consistency, and we compared their choices with the evaluator's pairwise winners.
| Weather | Evaluator–human agreement on the pairwise winner ↑ |
|---|---|
| Rain | 78.00% |
| Fog | 83.00% |
| Snow | 75.00% |
Q4. Independent gains from AW-Weather conditioning
With the same Wan backbone, adding an independent Weather AdaLN path improves VLM-Weather from 88.5 to 91.1, VLM-Technical from 79.7 to 82.8, and CLIP-S from 0.751 to 0.763. This comparison evaluates weather tokens alone against weather tokens with Weather AdaLN; full results are in Common Response C1.
With weather tokens alone, weather conditions and text share cross-attention. We observed incomplete weather removal or insufficient responses to the target intensity. Weather AdaLN injects source intensity, target intensity, and their difference through an independent path, improving these behaviors.
We separately ablate real-domain identity-mapping supervision using the same 15,000 training updates: aesthetic quality increases from 45.31 to 46.92, and imaging quality from 66.29 to 67.20. We will add the conditioning-path and real-domain training comparisons separately in the revision.
W5 / Q5. Reproducibility
Planned resources include the adapted VAE checkpoint, AW-Weather and AW-Harmony weights, releasable data and corresponding masks, data-generation and filtering scripts, and training, inference, and evaluation configurations. These resources are scheduled for release after paper acceptance and completion of the company's internal review.
We will also provide the filtering thresholds actually used, the filtering order and parameter settings, scene-level splits, and sample indices. The revised reproducibility statement will list the resources and release arrangements.
Among the public driving-video datasets we examined, adverse-weather samples were predominantly rainy. We did not find sufficient snow and fog videos meeting the task requirements, so we selected the corresponding videos from an internal database. The scope of releasable data will be specified with the resource list.
We thank the reviewer for the constructive feedback. We address each concern below.
To examine potential structural bias from generated inputs, we evaluate both training-pair quality and cross-source asset adaptation.
Preservation of training assets. Comparing temporally aligned original adverse-weather assets with pseudo-clear assets gives local DINOv2 of 0.7723 and CLIP-S of 0.8428. Asset-level VLM Preservation uses gpt-5.6-sol to assess identity, shape, color, position, and scale, ignoring weather appearance changes and judging only content visible in both crops. The normalized mean score is 91.8/100.
Effect of upstream filtering on downstream training. We separately train two AW-Weather Stage 2 models using filtered and unfiltered real-domain pairs. Each model then generates pseudo-clear assets to construct its own set of self-insertion pairs, and we train a separate AW-Harmony model on each set. With unfiltered upstream pairs, paired-track FVD increases from 15.1 to 19.7 and open-set Same-Weather decreases from 72.4% to 71.3%. This experiment examines how upstream pair filtering affects subsequent harmonization training outcomes.
Cross-source asset adaptation and preservation. Beyond training-pair checks, Track B inserts independently sourced assets into real adverse-weather backgrounds without using AW-Weather outputs as references. The added evaluation uses pre-harmonization assets as content references, with the same local crop expanded by 10% around the asset bounding box. AW-Harmony achieves local DINOv2 of 0.8244, CLIP-S of 0.8757, and Same-Weather of 72.4%.
These results support transfer of the adaptation ability learned through self-insertion supervision to cross-source assets and also evaluate content preservation in the final edited outputs. We report training-pair preservation separately from cross-source output preservation; the full method comparison is in Q3.
Table 7 already includes a mask-conditioned Wan baseline without the Weather Context Adapter, using standard ControlNet-style background conditioning. The comparison keeps the training pairs, adapted VAE, training budget, loss, and backbone unchanged. WC is 0.492 with the full Adapter versus 0.465 for the control; Same-Weather is 72.4% versus 68.4%. This replacement ablation directly examines the gains from the specialized context-adaptation design over general conditioning injection.
We also added same-pair training of existing models. Both DiffusionHarmonizer and VACE improve after training on the same pairs, while AW-Harmony still achieves lower FVD and higher open-set weather consistency. DiffusionHarmonizer and AW-Harmony both use 30,000 updates. Fine-tuning reduces DiffusionHarmonizer's FVD from 29.9 to 19.3 and increases open-set Same-Weather from 58.9% to 66.9%. VACE, using the same Wan backbone and adapted VAE, achieves FVD of 18.3 and open-set Same-Weather of 69.9% after fine-tuning on the same pairs, compared with 15.1 and 72.4% for AW-Harmony.
Full results and training settings are provided in C2.