acceptodds
Under review as a conference paper at ICLR 2027

WorldPortal: Recast Residual Correction for Instruction-Guided Video Background Replacement and Relighting

Abstract

Instruction-guided video background replacement and relighting require a model to synthesize a new environment and plausible illumination while preserving the identity, structure, texture, and motion of the foreground subject. Existing methods struggle to balance these objectives: training-free pipelines rely on rigid structural conditions that restrict natural lighting changes, whereas directly trained models degrade foreground structures and details. We present WorldPortal, a unified video editing framework based on Recast Residual Correction. During inference, a forward branch first predicts the video under the target background and illumination. A backward recasting branch then places the edited foreground state back into the source background domain, where its discrepancy from the source trajectory primarily captures foreground discrepancies introduced by editing. Transferring this residual to the target-domain prediction restores foreground details. To support both branches, we further introduce a joint VLM-DiT architecture that adaptively parses the instruction and input video into semantic-, foreground-, and background-centric token groups, providing flexible guidance. Experiments demonstrate that WorldPortal achieves superior overall performance in instruction alignment, lighting quality, structural fidelity, and fine-detail preservation, while introducing only modest additional inference cost.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Flow matching, Video editing, Video relighting

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_G19MOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses instruction-guided video background replacement and foreground relighting. The core idea is Recast Residual Correction. For each denoising step, a forward branch produces the target-background and illumination proposal, a backward recasting branch places the edited foreground state back into the source background domain. The difference between this recast prediction and the analytically aligned source trajectory is treated as an estimate of the foreground degradation caused by the edit and added back to the target proposal. Both branches are operating modes of a shared Wan2.2 5B DiT, conditioned by Qwen3VL-Embedding-8B through learned semantic-, foreground-, and background-centric query tokens that replace the hand-crafted mask, edge, and depth conditions of FlowPortal. Evaluation is on Sparkle-Bench against Kiwi-Edit, Sparkle, Lumen, and FlowPortal, where the method is best on 8 of 9 metrics.

Critical strengths
  1. The recast idea is well motivated, and the delta over FlowPortal is unusually clearly drawn. FlowPortal's correction is a source-reconstruction residual that never observes the edited state, which is why it needs a foreground mask, Canny/HED/depth controls, and high-frequency transfer — exactly the constraints that suppress natural lighting responses. WorldPortal's recast branch observes the current edit and estimates its actual degradation. The gains also land where the mechanism predicts.
  2. The practical profile is attractive. Input is a single instruction, with no foreground mask, no source or target captions, and no hand-built structural conditions. Qualitatively, the outputs show lighting responses such as shadow direction, highlight placement, and color cast that the constrained training-free pipeline does not produce.
Critical weaknesses
  1. The backward recasting branch is trained on a background reference whose construction is never specified. The other conditioning slot has the same problem: at training the foreground reference is a clean paired video, while at inference it is the one-step clean estimate, which is close to noise early in sampling. The influence of this gap is not discussed.
  2. The correction is applied without ever being checked. Equation 13 adds the full-frame residual unmasked, at unit weight, and reused for five consecutive steps, where FlowPortal applies it under a foreground mask. Nothing measures whether the residual actually stays in the foreground region, and no ablation covers a residual weight or the reuse interval. The concern is that the residual is computed in the source domain but applied to a target-domain proposal. The difference measures degradation only if the recasting branch first removes the intended relighting; otherwise the correction brings source appearance back and undoes the relighting it was asked to produce. Nothing tests which of the two happens, and the ablation is too thin to decide it either, with only two variants and no VLM metrics.
Questions for authors
  1. What exactly generates the background reference, under which instruction, and with what filtering? Please also validate the recasting branch on its own: with no edit applied, does the residual go to zero, and how faithfully does the branch place a given foreground into a specified background domain?
  2. Report how much of the residual energy falls inside versus outside the foreground mask, and ablate masked versus unmasked correction, a residual weight, and the reuse interval. On instructions that do not entail a lighting change, does the residual still help, or does it start pulling the foreground back toward the source?
3 comments
AuthorsOpenReview ↗

Response to Reviewer G19M Part 1

We thank the reviewer for the thorough review, constructive comments, and valuable questions. We also appreciate the reviewer’s recognition of the well-motivated recasting mechanism, its clearly explained distinction from FlowPortal, the practical simplicity of instruction-only input, and the natural lighting responses demonstrated by our results. Below, we address each concern and answer all of the reviewer’s questions.

Q1. Background-reference construction

Due to space limitations, we described the reference's role in Appendix C.1 but did not fully explain how it was constructed. For each triplet, we construct the background reference using one of two methods, selected with equal probability:

  1. We extract the foreground mask from the source video and use Wan2.1-VACE 14B [1] to inpaint away the foreground, then paste the original background region back into the result.
  2. We apply SDEdit with Wan2.2 to the source video, that is, partially adding noise and then denoising to simulate foreground changes, then paste the original background region back into the result.

Both methods produce a reference with the same background as the source video but a different foreground, providing the recasting branch with a reference for returning to the source background. We will include the detailed construction procedure in the revision.

During training, foreground variations are simulated through this preparation process, whereas at inference the source video itself serves as the source-domain reference. Although the references are obtained differently, the recasting objective is consistent: transferring the given foreground back into the specified source background. Our experimental results support the effectiveness of this approach.

Q2. Identity recast without editing

We conducted an experiment using the source video as both foreground and background reference, with the null prompt, and compare the recast proposal against the analytical source trajectory at matched timesteps. The residual norm is 0.006 RMS per latent element, compared with 0.041 RMS per latent element under actual edits, averaged per video over matched timesteps and then over the evaluation set. An accurate recasting model should produce a small identity residual, but exact zero is not guaranteed by a learned predictor. This measures the recasting-error floor.

Q3. Standalone recasting accuracy

We also evaluate recasting independently on held-out triplets with a known target video, using the original Identity, Structure, Detail, and VLM-Quality metrics.

MethodIdentity ↑Structure ↑Detail ↑VLM-Quality ↑
WorldPortal w/o RRC (forward only)0.5510.60830.8623.831
WorldPortal (full)0.7570.71733.3443.975
Standalone recasting0.7320.68432.1863.998

In WorldPortal, the recasting branch primarily serves to model the degradation introduced during forward editing. Because forward editing and backward recasting share the same model parameters, the backward process exhibit degradation similar to that of the forward process. Computing a residual between the recast prediction and the original video in the source domain therefore provides a correction signal that helps recover structural details lost during forward editing.

Continuing response in the thread (References are included in the last part of the thread)

AuthorsOpenReview ↗

Response to Reviewer G19M Part 2

Q4. Localization, masking, and residual leakage

The recasting branch is trained to combine a foreground reference with the specified background and illumination. Comparing its prediction with the aligned source trajectory is intended to suppress the desired domain change before estimating foreground discrepancies.

Our analysis finds 83.73% of residual energy inside the foreground mask and 16.27% outside. This supports spatial localization but does not, by itself, distinguish structural error from lighting differences or establish negligible background effects. We will specify the energy definition, aggregation, and mask area, and compare rtr_t with M⊙rtM\odot r_t under identical sampling settings.

CorrectionDetail ↑Identity ↑CLIP-T ↑VLM-Inst. ↑VLM-Quality ↑
No correction30.8620.5510.2403.8963.831
Foreground-masked33.2110.7520.2534.0153.960
Unmasked33.3440.7570.2524.0183.975

The masked/unmasked pixel difference is 0.0038 over the full frame and 0.0031 over the background, measured as RGB MAE on [0, 1]. Since most of the residual energy is concentrated within the foreground mask and applying the mask produces negligible differences in the reported metrics, RRC can operate without an explicit foreground mask at inference, avoiding additional mask estimation while maintaining comparable editing performance.

Although most residual energy is concentrated in the foreground, as you noted, the residual may still carry illumination information from the source video. We claim that, most of the recast residual primarily compensates for structural and fine-detail degradation. We examine its energy distribution across spatial-frequency bands, ordered from low to high:

Spatial-frequency band (low → high)Share of total residual energy (%)
0–10%0.43%
10–20%0.91%
20–30%1.37%
30–40%1.85%
40–50%2.64%
50–60%3.18%
60–70%4.06%
70–80%5.29%
80–90%17.46%
90–100%62.81%

The concentration of residual energy at high frequencies suggests that the residual primarily provides structural and fine-detail compensation, with limited transfer of source illumination to the target video.

Q5. Correction weight

Unit weight follows the condition-consistency principle underlying residual correction in FlowPortal [2]: adding the full residual restores the source-domain prediction to the original video trajectory. We evaluate zt−Δt=zˉt−Δtedit+λrtz_{t-\Delta t}=\bar z^{\mathrm{edit}}_{t-\Delta t}+\lambda r_t with different λ\lambda s .

Weight λ\lambdaIdentity ↑Detail ↑VLM-Inst. ↑VLM-Quality ↑
00.55130.8623.8963.831
0.50.70632.5493.9713.947
10.75733.3444.0183.975
1.50.76533.4253.9413.925

An overly small λ\lambda may leave details insufficiently restored, whereas an overly large λ\lambda may impose excessive constraints on details and push the sampling trajectory out of distribution, reducing visual quality.

Q6. Refresh interval

The default interval K=5K=5, adopted from FlowPortal, is intended to balance correction freshness and computation. We compare refresh intervals under the same 50-step sampler. With forward CFG and no backward CFG, the DiT pass count is 100+⌈50/K⌉100+\lceil50/K\rceil.

Interval KKDiT passesDetail ↑Identity ↑Structure ↑VLM-Inst. ↑VLM-Quality ↑Average runtime per video(s) ↓
115033.4740.7600.7114.0203.986240
212533.3990.7530.7144.0213.980204
511033.3440.7570.7174.0183.975189
1010532.9500.7340.6983.9903.935174

More frequent refreshes provide modest quality gains at a higher computational cost, while less frequent refreshes improve efficiency but may weaken correction. The default interval offers a balance between editing quality and efficiency.

Q7. Broader ablations and VLM metrics

We additionally evaluate more ablation variants using VLM metrics:

VariantVLM-Inst. ↑VLM-Const. ↑VLM-Quality ↑
w/o RRC3.8873.7193.831
w/o VLM conditioning and tokens3.5493.4783.504
Foreground-masked RRC4.0074.0613.960
Source-reconstruction residual3.9933.9893.936
Full model4.0184.0823.975

References

[1] Jiang et al. VACE: All-in-One Video Creation and Editing. ICCV, 2025.

[2] Gao et al. FlowPortal: Residual-Corrected Flow for Training-Free Video Relighting and Background Replacement. 2025.

Thanks for the authors feedback. It is very helpful to see the details of background-reference constructions. The comprehensive ablation about different variants of the proposal and the hyper-parameters. My concerns have been addressed and I will keep my initial rating for acceptance.

Reviewer_j3hcOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes WorldPortal, a residual correction method for instruction-guided video background replacement and foreground illumination adaptation. The main component of WorldPortal is the Recast Residual Correction mechanism, which uses a forward branch and a backward recasting branch to compute a residual. Specifically, WorldPortal maps the current state from the editing trajectory back to the source's background, computes the residual against an analytical source trajectory, and adds this residual to the editing proposal trajectory to correct foreground distortions. This is implemented via a shared video DiT and VLM architecture, where the editing and recasting branches are supported through semantic/foreground/background-centric token groups. The paper evaluates the framework on Sparkle-Bench, where WorldPortal improves over four baselines.

Critical strengths
  1. The trade-off between flexible relighting editing and foreground preservation in video models is a practically important problem.
  2. The reported results for WorldPortal are positive across eight of nine metrics and ablations show that removing the recasting mechanism degrades identity, structure, and detail consistency metrics.
Critical weaknesses
  1. My main concern is that the benefits of conditioning the correction on the current edited state are not sufficiently isolated. Table 2 shows that recast correction improves performance over using no correction. However, a matched source-reconstruction residual baseline would establish whether observing the current edited foreground provides an additional benefit. Also, the assumption that the computed residual captures foreground degradation is plausible, but insufficiently validated. Using this method with imperfect recasting could also introduce background or illumination artifacts.
  2. In the main experiment, FlowPortal remains competitive and the gap is small on multiple metrics. If the problem with FlowPortal is that its explicit structural contraints can limit the flexibility of lighting response or produce overly strong illumination effects, it is possible that simply tuning FlowPortal's structural constraints via hyperparameters improve the results. Evaluation is also limited to Sparkle-Bench after training on Sparkle data, so generalization beyond a single benchmark is uncertain. Hence, whether WorldPortal has a distinct advantage that survives different datasets or simple tuning of training-free methods is insufficiently addressed.
Questions for authors
  1. Could the authors provide a comparison between recasting the current edited state (WorldPortal) and a simpler source-reconstruction residual within the WorldPortal architecture? Please also assess whether errors in recasting introduce background or illumination artifacts through the transferred residual.
  2. Are the gains be specific to the choice of pretrained models (Wan2.2-T12V-5B/Qwen3VL-Embedding-8B VLM) used in the experiments? Also, for FlowPortal, were its control strengths tuned and does WorldPortal retain its advantages in different settings?
4 comments
AuthorsOpenReview ↗

Response to Reviewer j3hc Part 1

We thank the reviewer for the thorough review, constructive comments, and valuable questions. We also appreciate the reviewer’s recognition of the practical importance of balancing relighting flexibility with foreground preservation, the positive results across eight of nine metrics, and the improvements in identity, structure, and detail consistency provided by recasting. Below, we address each concern and answer all of the reviewer’s questions.

Q1. Matched source-reconstruction control

We agree that the original no-correction ablation does not isolate the value of observing the current edited state. We add a control replacing only the recasting foreground reference z^0edit\hat z_0^{\mathrm{edit}} with the source video z0sz_0^s. Thus, the control measures source reconstruction error without observing foreground degradation in the current edit.

CorrectionDetail ↑Identity ↑Structure ↑VLM-Quality ↑
None30.8620.5510.6083.831
Source-reconstruction residual32.2480.6970.6863.930
Current-edit recast residual33.3440.7570.7173.975

The final two rows directly test the additional value of edit-dependent recasting, without differences between complete architectures confounding the comparison. Therefore, conditioning the correction on the current edited state improves foreground preservation and visual quality beyond source-reconstruction correction alone.

Q2. Residual validity and artifact propagation

Recasting places the edited foreground back into the source environment and illumination, enabling comparison with the source trajectory to estimate foreground discrepancies introduced by editing. To address your concern about possible background or illumination artifacts, we examine the residual’s spatial distribution and compare masked and unmasked correction below.

Our analysis finds 83.73% of residual energy inside the foreground mask and 16.27% outside. To assess the effect of the background component, we compare foreground-masked and unmasked correction under the same sampling settings:

CorrectionDetail ↑Identity ↑CLIP-T ↑VLM-Inst. ↑VLM-Quality ↑
No correction30.8620.5510.2403.8963.831
Foreground-masked33.2110.7520.2534.0153.960
Unmasked33.3440.7570.2524.0183.975

The pixel difference between masked and unmasked outputs is 0.0031 in the background and 0.0038 over the full frame, measured as RGB MAE on [0, 1]. Since most of the residual energy is concentrated within the foreground mask and applying the mask produces negligible differences in the reported metrics, these results support mask-free correction while maintaining comparable editing performance.

Q3. Dependence on pretrained models

Our formulation is not tied to a specific checkpoint. The comparison below varies the video model with matched evaluations with and without Recast Residual Correction (RRC).

Video backbone / VLMRRCIdentity ↑Structure ↑VLM-Quality ↑
Wan2.2-TI2V-5B / Qwen3VL-Embedding-8BNo0.5510.6083.831
Wan2.2-TI2V-5B / Qwen3VL-Embedding-8BYes0.7570.7173.975
Wan2.1-Fun-V1.1-1.3B-Control / Qwen3VL-Embedding-8BNo0.5090.5753.700
Wan2.1-Fun-V1.1-1.3B-Control / Qwen3VL-Embedding-8BYes0.6740.6663.812

Given the substantial training cost, we currently report only the results of replacing the video diffusion model. We will provide additional results on VLM architectures and parameter initialization in the revision.

Continuing response in the thread (References are included in the last part of the thread)

AuthorsOpenReview ↗

Response to Reviewer j3hc Part 2

Q4. FlowPortal control strengths

FlowPortal [1] requires extracting a structural video from the source video and removing its background before using it as a control signal. This preprocessing relies on external models and imposes a strong constraint derived from the source appearance. Such a constraint can conflict with edits that require changes to foreground appearance, particularly when relighting should alter shading or shadows. Stronger control may therefore preserve source details at the expense of the requested edit.

To examine this trade-off, we compare different FlowPortal control strengths and evaluate foreground preservation, instruction adherence, and visual quality, as shown below.

ConfigurationStructure ↑Identity ↑VLM-Inst. ↑VLM-Quality ↑
FlowPortal, control 1.00.7010.6893.7813.910
FlowPortal, control 0.50.6810.6633.8333.924
FlowPortal, control 1.50.7030.6963.7353.876
WorldPortal0.7170.7574.0183.975

These results show that our recasting mechanism automatically identifies and compensates for foreground degradation, enabling video generation without additional structural control. This gives the editing process greater flexibility while better preserving the quality of the generated videos.

Q5. Generalization beyond Sparkle-Bench

We report the additional evaluation on an IVE-Bench background-editing subset [2], containing 100 videos selected randomly.

MethodDetail ↑Identity ↑CLIP-T ↑VLM-Quality ↑
Sparkle28.9320.4450.2353.701
FlowPortal31.4670.6690.2233.848
WorldPortal w/o RRC30.0150.5270.2323.756
WorldPortal32.5750.7290.2373.860

The within-model comparison indicates RRC remains useful under dataset shift.

References

[1] Gao et al. FlowPortal: Residual-Corrected Flow for Training-Free Video Relighting and Background Replacement. 2025.

[2] Chen et al. IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessment. 2025.

We sincerely appreciate the reviewer’s constructive feedback. We hope that our response has adequately addressed your concerns. As the discussion period is drawing to a close, we would like to offer a gentle final reminder. We fully respect your judgment and simply wish to ensure that no concerns remain unresolved. Please feel free to let us know if any further clarification would be helpful.

I thank the authors for their detailed response and new experiments. Their response has sufficiently addressed my questions and concerns. The matched source-reconstruction control shows that recasting uniformly improves all four metrics in Table 4, although the majority of the gains seem to come from source-reconstruction residual. My concerns regarding artifact propagation, robustness across video backbones, practical advantage over FlowPortal, and generalization to a different benchmark have all been addressed sufficiently. Thus, I raise my score and I do not have additional questions.

Reviewer_xD4kOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes WorldPortal for instruction-guided video background replacement and foreground relighting. It has two parts. The first is Recast Residual Correction (RRC), an inference-time scheme in which a backward “recast” branch maps the current edited foreground into the source background domain; its deviation from the analytic source trajectory is then added to the forward update. The second is a shared DiT that performs forward editing and backward recasting using a VLM with learnable queries. The problem is well motivated and significant gains are reported.

Critical strengths

The idea is intuitive and sound. The authors observe that foreground degradation from direct editing is hard to measure in the target domain, where lighting has legitimately changed. They move the problem back to the source domain instead: recasting the current edited state into the source background makes the degradation directly comparable with the known source trajectory. This gives an edit-aware correction signal at every denoising step. The experimental results demonstrate higher performance of the proposed method.

Critical weaknesses

The major weakness is that the ablation study is insufficient to demonstrate the effectiveness of all proposed components. The conditioning design is ablated only as a whole, so the source of its gains is unclear. The paper adds a VLM condition extractor, three groups of learnable queries, and three groups of learnable null queries. Table 2 removes all of these together in one ablation in Table 2. None of the semantic-, foreground- or background-centric token groups, nor the null-query mechanism, is validated on its own. With this much extra trainable capacity, the gain may come simply from more learnable parameters rather than from the proposed role-specific token decomposition.

Questions for authors
  1. As mentioned in the weakness, can the authors ablate each component separately? Please also show whether the foreground-centric and background-centric tokens actually affect foreground and background quality respectively. Otherwise the gain cannot be distinguished from added capacity.
  2. In Table 2, "w/o Recast” only disables the backward branch at inference while reusing jointly trained weights. What is the performance of a model trained with the forward editing branch alone?
  3. In Table 2, "w/o Recast” lowers CLIP-T from 0.252 to 0.240. However, Recast Residual Correction is meant to correct only the foreground, while the prompt mainly describes the background and lighting, so this gain needs explanation. Does the unmasked residual alter the background?
  4. The authors should provide more examples of the source video, the forward-edited video, the backward-recast video, and the corrected video. These would verify the proposed method works as expected.
1 comment
AuthorsOpenReview ↗

Response to Reviewer xD4k

We thank the reviewer for the thorough review, constructive comments, and valuable questions. We also appreciate the reviewer’s recognition of the clear presentation, the intuitive and well-motivated recasting mechanism, and the performance improvements demonstrated by our experiments. Below, we address the concerns about the ablation study and answer each of the reviewer’s questions.

Q1. Individual components, foreground/background roles, and capacity

We agree that the original combined ablation does not isolate individual token groups or exclude the effect of additional capacity. Following your suggestion, we first report separate foreground- and background-token ablations. Foreground tokens guide subject preservation in forward editing, while background tokens specify the source environment and illumination during recasting. We also include a No VLM variant, which removes the VLM extractor and its associated queries and connectors while retaining Recast Residual Correction (RRC). We use Identity and Structure [1] to assess foreground preservation, and CLIP-T and VLM-Quality to assess overall instruction alignment and visual quality.

VariantTrainable paramsIdentity ↑Structure ↑CLIP-T ↑VLM-Quality ↑
Full model5.18B0.7570.7170.2523.975
w/o foreground tokens5.18B0.6810.6740.2443.906
w/o background tokens5.18B0.7100.6920.2373.851
No VLM5B0.4230.5460.2243.504

Given the substantial training cost, we currently provide the results above. We plan to include semantic-token, null-query, and parameter-matched generic-token controls in the revision.

Q2. Forward-only training

We agree that the original “w/o Recast” disables recasting only at inference. We add a model trained exclusively for forward editing:

Training / inferenceDetail ↑Identity ↑Structure ↑CLIP-T ↑
Forward-only / forward-only30.4100.5280.6550.229
Joint / forward-only30.8620.5510.6080.240
Joint / forward + RRC33.3440.7570.7170.252

The first two rows isolate the training contribution, while the final two isolate inference-time correction using the same jointly trained checkpoint.

Q3. CLIP-T improvement and background effects

CLIP-T is computed over the entire frame, so it also reflects the foreground and its interaction with the scene. By preserving foreground structure and details, RRC can improve overall image–text alignment even when the instruction mainly describes the background and lighting.

Our experiment indicates that the residual is concentrated in the foreground, with 83.73% of its energy lies inside the foreground mask and 16.27% outside. Since the background component is nonzero, we compare correction restricted to the foreground mask with full-frame correction under the same sampling settings:

CorrectionDetail ↑Identity ↑CLIP-T ↑VLM-Inst. ↑VLM-Quality ↑
No correction30.8620.5510.2403.8963.831
Foreground-masked33.2110.7520.2534.0153.960
Unmasked33.3440.7570.2524.0183.975

The pixel difference between masked and unmasked outputs is 0.0031 in the background and 0.0038 over the full frame, measured as RGB MAE on [0, 1].

Since most of the residual energy is concentrated within the foreground mask and applying the mask produces negligible differences in the reported metrics, RRC can operate without an explicit foreground mask at inference, avoiding additional mask estimation while maintaining comparable editing performance.

Q4. Complete intermediate examples

We will add source, forward-edited, backward-recast, and corrected examples to the revised manuscript.

References

[1] Gao et al. FlowPortal: Residual-Corrected Flow for Training-Free Video Relighting and Background Replacement. 2025.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global Response

Global Response

We thank all reviewers for their careful reading and constructive feedback. We appreciate their recognition of the practical importance of balancing foreground preservation with flexible relighting (Reviewer j3hc), the intuitive and well-motivated recasting mechanism (Reviewers xD4k and G19M), and the improvements in foreground fidelity demonstrated by our experiments (all three reviewers). We also thank Reviewer G19M for highlighting the practical simplicity of instruction-only editing and the natural lighting responses of our results.

We provide detailed responses in the individual reviewer threads. Here, we summarize the central contribution and the additional evidence addressing the main concerns.

Core contribution. WorldPortal introduces Recast Residual Correction (RRC), which observes the current edited foreground and maps it back into the source environment. Comparing the recast prediction with the known source trajectory provides a correction signal for foreground degradation. This allows the model to preserve foreground structure and details while accommodating changes in background and illumination, without requiring explicit foreground masks or hand-crafted structural controls at inference.

Additional experimental evidence. Following the reviewers' suggestions, we have expanded our evaluation in three directions:

  • Isolating the contributions of recasting and conditioning. We add a matched source-reconstruction residual baseline within WorldPortal, separate foreground- and background-token ablations, a model trained only for forward editing, and additional VLM metrics. The matched residual comparison shows that observing the current edited state improves foreground preservation and visual quality beyond source-reconstruction correction alone.
  • Examining residual behavior and correction settings. We clarify background-reference construction and report identity recasting, standalone recasting, residual localization and frequency analysis, masked versus unmasked correction, and correction-weight and refresh-interval ablations. These results support foreground-focused correction and comparable performance without an explicit inference mask, while characterizing the trade-offs between correction strength, editing quality, and computational cost.
  • Testing performance beyond the original setting. We evaluate an alternative video diffusion backbone, different FlowPortal control strengths, and an additional IVE-Bench subset. RRC remains beneficial with the alternative backbone and on the additional benchmark, and WorldPortal retains its advantages over the evaluated FlowPortal control settings.

We will incorporate these clarifications and additional results into the revised manuscript, together with the further component studies and intermediate examples discussed in our individual responses. We thank the reviewers again for helping us strengthen the experimental validation and presentation of WorldPortal.

AuthorsOpenReview ↗

First Revision Update

We have uploaded the first revised version of our manuscript, incorporating additional experiments and clarifications from our rebuttal. All new content is highlighted in yellow. Most of the additions are in the appendix, including further implementation details and ablation studies. The residual localization and foreground-masking experiments have also been added to the main text.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.