acceptodds
Withdrawn from ICLR 2027

MeteoVerse: Unified Weather-Controllable Video World Model

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Video world models aim to predict future content from an observed scene while following prescribed camera motion. Real-world scene evolution is determined not only by changes in viewpoint and object dynamics, but also by environmental conditions such as weather, which can substantially alter scene appearance and visibility. Modeling such realistic weather evolution is challenging because the required weather modification depends jointly on the observed and desired weather states. Depending on their relation, the model may need to preserve, introduce, or remove a weather effect. Existing video world models typically leave this weather transition implicit, forcing the generation backbone to infer weather evolution together with scene dynamics and camera motion, which leads to imprecise weather control. To address this limitation, we propose MeteoVerse, a unified weather-controllable video world model that generates future videos from a single sunny or adverse-weather image, conditioned on a weather-free scene description, a target-weather instruction, and a camera trajectory. Rather than conditioning only on the desired weather, MeteoVerse explicitly estimates the observed and target weather states and represents the required weather transition. A transition-aware mixture of weather experts (MeteoMoE) then translates this transition into category-specific residual weather features, unifying weather preservation, introduction, and removal while enabling fine-grained control over introduced weather intensity. We further construct the MeteoVerse dataset with over 50K real-world weather video clips, generated sunny counterparts, disentangled scene and weather descriptions, weather-intensity annotations, and camera trajectories. Extensive quantitative and qualitative experiments demonstrate substantially improved weather controllability while retaining competitive scene consistency and camera-control performance.

Primary area:
generative models
Keywords:
Weather Synthesis, Video World Model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ySwvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MeteoVerse, a weather-controllable video generation model which takes a single image, a weather-free scene description, a target-weather instruction, and a camera trajectory. Built upon LingBot-World, it freezes the original backbone weights and trains LoRA adapters together with a weather-conditioning module, MeteoMoE. A weather-state predictor estimates source and target rain, snow, and fog intensities. Their difference modulates category-specific weather features injected into the video backbone. The authors also construct a dataset from over 50K real adverse-weather clips and generated sunny counterparts. Experiments evaluate weather preservation, introduction, and removal.

Critical strengths
  1. MeteoVerse supports adverse-weather inputs and weather removal. It unifies weather preservation, introduction, and removal within one model.
  2. MeteoVerse achieves great Weather Alignment performance on both qualitative results and quantitative scores, in introduction and removal tasks
Critical weaknesses
  1. The metrics of Weather Alignment: In Table 2, target-weather text conditioning achieves 80.85% Weather Alignment, nearly matching the proposed transition representation at 81.00%. Although VLM and human scores improve, no uncertainty estimates are provided. Intuitively, the parameter-based guidance should provide a more smooth control to the results and the images in Figure 5 can support this conclusion. However, the value itself does not strongly support it since the improvement is limited.
  2. The quality of the pseudo-paired training data is insufficiently validated. Sunny counterparts are generated by editing the first adverse-weather frame and synthesizing a new video with LingBot-World. This procedure does not ensure that object motion, geometry, and newly visible content remain aligned with the original video. The paper describes filtering but provides no quantitative audit of pair correspondence, scene preservation, or filtering effectiveness. Consequently, the extent to which supervision isolates weather changes from unrelated content changes remains unclear.
  3. The demonstrated transition coverage is limited. Training and evaluation focus on clear-to-adverse, adverse-to-clear, and preservation settings. The adverse-to-adverse benchmark evaluates preservation rather than changes between weather categories. There is no systematic evaluation of rain-to-snow transfer, partial intensity reduction from an adverse input, or selective removal within mixed weather. These cases would directly test the proposed multicomponent transition formulation.
  4. Fine-grained weather control lacks quantitative validation. Intensity control is primarily demonstrated through selected visual examples. Weather Alignment measures the presence of the requested condition, while the VLM score combines several qualitative criteria; neither directly measures achieved intensity. The predictor’s intensity MAE does not establish that generated videos follow the requested intensity. Calibration, monotonicity, and cross-category interference remain unmeasured.
Questions for authors

Please refer to my weaknesses and concern about the following questions:

  1. How are generated sunny image/video pairs assessed for preservation of geometry and object motion? Please provide filtering acceptance rates and a quantitative or human audit of correspondence quality. How are camera trajectories assigned to the cross-weather training pairs after re-estimation?
  2. How does MeteoVerse compare directly with the Nano Banana 2 → LingBot-World pipeline used to construct sunny training targets, particularly for weather removal, scene preservation, and inference cost?
  3. Can the model perform cross-category transitions, reduce existing weather intensity, or independently manipulate mixed weather? For example, translate a rainy weather to a snowy weather Please provide quantitative intensity-following and monotonicity results across multiple scenes.
Reviewer_rkZEOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MeteoVerse, a weather-controllable video world model that supports weather preservation, introduction, and removal from a single image. The method estimates the observed and target weather states and uses a transition-aware mixture of weather experts (MeteoMoE) to inject weather-specific residual features into a frozen LingBot-World backbone. The authors also construct a dataset of over 50K weather video clips with generated sunny counterparts, weather annotations, and camera trajectories.

The work is closely related to Holo-World (Yin et al., 2026), which previously introduced weather-controllable video world modeling from a single image. MeteoVerse extends the functional setting to weather removal and preservation, but its methodological contribution beyond this prior work remains unclear.

Missing required AI use statement. The submitted manuscript does not contain the mandatory AI use statement required by ICLR 2027. This is particularly relevant because the paper explicitly reports using generative AI tools during dataset construction and annotation. I recommend that the Area Chair verify whether the required disclosure was submitted separately and request clarification from the authors regarding compliance with the conference policy.

Critical strengths
  1. The paper extends weather-controllable video world modeling to support both sunny and adverse-weather inputs within a single framework. Explicit weather transition modeling also provides fine-grained control over the intensity of introduced weather effects.
  2. The authors construct a dataset containing over 50K real-world weather video clips with generated sunny counterparts and weather-intensity annotations. The experiments cover three weather-control settings and evaluate video quality, camera control, and weather alignment.
Critical weaknesses
  1. Holo-World (Yin et al., 2026) already introduced single-image weather-controllable video world modeling through disentangled scene and weather conditioning, residual weather adapters, and pseudo-paired training data. MeteoVerse follows a similar framework, with MeteoMoE mainly replacing the adapters with category-specific experts and transition-weighted feature fusion. While weather preservation and removal extend the functionality, the methodological novelty remains limited. More importantly, Holo-World is absent from the quantitative comparisons, making it difficult to assess the improvement over the most relevant prior work.

  2. The dataset relies on generated sunny counterparts without adequately evaluating their geometric, semantic, or temporal consistency with the original adverse-weather videos. Such discrepancies may introduce supervision noise unrelated to weather changes and affect generalization.

  3. Weather annotations are generated using Qwen3-VL models, while Weather Alignment and VLM Evaluation also rely on the Qwen model family. This may introduce correlated annotation and evaluation biases. Independent evaluation using other VLMs or human-verified annotations is needed.

Questions for authors

Q1. Can the authors provide a direct comparison with Holo-World under a common evaluation protocol, particularly for weather introduction and scene preservation? How does MeteoMoE improve upon the existing weather-control adapters?

Q2. Beyond extending weather introduction to preservation and removal, what is the main methodological novelty of MeteoMoE compared with standard category-specific adapters and conditional feature fusion?

Q3. How consistent are the generated sunny counterparts with the original adverse-weather videos in terms of scene geometry, object motion, and temporal structure? How does pseudo-pairing noise affect training and generalization?

Q4. Can the authors evaluate weather alignment and generation quality using an independent VLM outside the Qwen family? How are potential annotation and evaluation biases controlled?

Q5. Does MeteoVerse support mixed weather conditions, such as rain with fog or snow with fog? If so, can the authors provide quantitative results for simultaneous weather effects and transitions?

Q6. Can the authors provide detailed dataset statistics, including video sources, scene categories, camera-motion distributions, and weather-intensity distributions?

Q7. How were the continuous weather-intensity annotations validated against independent human judgments? Does the reported weather-state prediction accuracy remain reliable with independently annotated ground truth?

Q8. Can the authors compare MeteoMoE against simpler weather-control adapters with matched trainable parameters and training data, separating the contribution of the transition representation from that of the additional LoRA adaptation?

Reviewer_2KKtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MeteoVerse, a camera-controlled image-to-video world model built on LingBot-World that can keep, add or remove rain, snow and fog. A LoRA-tuned Qwen3-VL-2B estimates a weather state (rain, snow and fog intensities in [0, 1]) for the input image and for the target-weather instruction, and the paper calls their difference the weather transition. A module called MeteoMoE turns the transition into features: in every DiT block, the video features attend to one set of learnable tokens per weather type, each result is scaled by the matching part of the transition, and the results are fused and added back. Training starts from over 50K real rain, snow and fog clips. Each kept clip gets a sunny version: its first frame is edited with Nano Banana 2, and LingBot-World turns the edited frame into a video. Pairing either first frame with either video gives four training pairs. The test set has 100 held-out scenes, each with a sunny or adverse input and target: keeping the weather is preservation, sunny to adverse is introduction, and adverse to sunny is removal. Five camera-controlled world models serve as baselines. Evaluation uses VBench, camera errors, a user study and a VLM judge, whose Weather Alignment score is the share of videos that show the requested weather.

Critical strengths
  1. I consider the MeteoVerse dataset as the main contribution. Paired videos of the same scene with and without bad weather are hard to collect, and the paper builds them at scale: over 50K real rain, snow and fog clips, each with a weather-free scene description, a separate weather description, per-type intensity labels, camera poses and a generated sunny version (Sec. 3.4). Pairing either first frame with either video (Eq. 8) then gives supervision for keeping, adding and removing weather in one model.

  2. The gains are large when the weather has to change. Weather Alignment is 89 for removal and 61 for introduction, while the best baseline reaches 38 and 29 (Table 1). In the supplementary videos, which cover every method on all 10 scenes in all four settings, the baselines mostly leave the input weather as it is, while MeteoVerse visibly adds or removes it. Replacing the predictor's weather states with the annotated ones changes the results little (Table 4), so the predictor is not a weak point.

Critical weaknesses
  1. To me the gains come from fine-tuning the backbone on the new paired data, not from the transition design. In the ablation (Table 2), a variant given only the target-weather text reaches nearly the same Weather Alignment as the full model (80.85 vs 81.00); on the other two weather scores the full model leads by 5.4 and 3.0 points. The untuned LingBot-World averages 45 on the same measure (Table 1). Table 3 only compares MeteoMoE with weaker ways of feeding the same transition, while removing the LoRA adapters on the backbone drops Weather Alignment to 52.02. The paper argues that leaving the transition implicit makes control imprecise (p. 2), but these results do not show it. The design is also narrow: everything is built for rain, snow and fog, and the instruction reaches the video model only as three numbers (Eq. 3), so hail, a sandstorm or a lighting change would need new experts, labels and paired data. The obvious baseline is also missing: edit the weather of the input image, then run LingBot-World. The sunny training videos were made this way (pp. 5-6).

  2. Three claims in the abstract and introduction go beyond the evidence. The paper frames the task as controlling "how weather evolves into the future" (p. 2), but nothing evolves over time. Every training target already shows the target weather from its first frame (Eq. 8), and in the introduction and removal videos (supplement; t=0 in Figs. 1 and 4) the first frame, which an image-to-video model normally copies from the input, already shows the target weather in full, and it stays roughly the same afterwards. Fine-grained intensity control is shown only in Fig. 5, on one scene per weather type, and no number checks whether the strength of the output follows the requested level. Camera control is called competitive, but the rotation error is 3.2 to 3.5, against 0.3 to 0.8 for GEN3C, Uni3C and NeoVerse (Table 1). It improves only on LingBot-World, its backbone, and on VerseCrafter.

Questions for authors

Q1. How does MeteoVerse compare with editing the weather of the input image (for example with Nano Banana 2, as you did to make the sunny versions) and then running LingBot-World on the same camera path? In Table 2, is the text-only variant plain LoRA fine-tuning of LingBot-World on the same data, or does it keep MeteoMoE?

Q2. Will the dataset be released, and where do the clips come from and under what licences? Statistics such as the number of clips per weather type and the intensity distribution would also help judge it.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.