acceptodds
Under review as a conference paper at ICLR 2027

ISI-WM: Interventional Selective Invariance for World Models

Abstract

Visual world models enable prediction and control directly from pixels, but reliable deployment requires representations that remain stable under irrelevant appearance changes without losing action-dependent dynamics. Existing robustness objectives typically align augmented views without identifying which variation is a nuisance or when distinct actions produce outcomes that must remain separable. We address this gap with Interventional Selective Invariance for World Models (ISI-WM), a training framework that suppresses background-induced variation while preserving distinctions associated with divergent action outcomes. A background intervention pairs observations of the same physical state rendered against different backgrounds and aligns their latent encodings. An action intervention forks alternative actions from a shared state, trains the latent rollout toward its executed branch, and activates separation and ranking only when the realized simulator outcomes diverge. Together, these signals optimize the existing RGB encoder and latent dynamics without extending the underlying architecture; simulator access is confined to training, and deployment remains RGB-only. Under a unified dynamic-background protocol, ISI-WM achieves stronger overall generalization to unseen backgrounds than the evaluated baselines, while ablations and latent-response analyses support the complementary roles of the background and action interventions. Code is available at https://anonymous.4open.science/r/isi-wm-code-E0AF.

Primary area:
reinforcement learning
Keywords:
world models, visual reinforcement learning, model-based reinforcement learning, out-of-distribution generalization, selective invariance, interventional training

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_KF8dOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ISI-WM, a training framework for learning visual world models that ignore background changes while preserving distinctions associated with divergent action outcomes. Built on TD-MPC2, it combines cross-background state alignment with outcome-gated action-fork learning, separation, and ranking objectives. Separation and ranking are activated when alternative actions produce sufficiently different simulator states. Evaluation on six DMC tasks shows improved average return under unseen video backgrounds, without adding trainable parameters or changing the RGB-only deployment interface.

Critical strengths

The strengths of the paper are:

  1. The insight about distinction between nuisance invariance and action sensitivity is well motivated. Aligning observations of the same physical state under different backgrounds gives a clear supervision signal for irrelevant variation. Conditioning separation on realized outcomes is also more principled than assuming that different commanded actions must always produce different states. The paper carefully describes these as controlled training interventions without claiming causal identification or a formal abstraction guarantee.
  2. The improvements over the TD-MPC2 model are substantial. On unseen backgrounds, the average return increases from 216 to 434, approximately doubling performance. Particularly large gains occur on Cup Catch, from 67 to 537, and Reacher Easy, from 231 to 630. The method also exceeds HRSSM’s macro-average of 278, although it does not outperform HRSSM on every task.
  3. The ablations provide useful evidence that the two supervision sources complement each other. In ablation study, unseen background macro-average returns are 214 without interventions, 294 with background alignment, 264 with action forks, and 433 with both. On Cup Catch, the combined method reaches 527, compared with 166 and 169 for the individual components. These results support the combined design beyond simply showing an improvement over the base model.
  4. As for the model size and latency, ISI-WM retains 4.86 million parameters, and median action latency remains essentially unchanged at 25.78 ms versus 25.85 ms for TD-MPC2, demonstrating the efficiency of the proposed method. The authors also report the costs rather than presenting the method as free: training time increases from 11.9 to 14.8 hours, and peak inference allocation rises from 66.9 to 97.4 MB.
Critical weaknesses
  1. Basically, ISI-WM receives auxiliary trajectories constructed through simulator replay, exact background correspondences, and state derived outcome labels. Its roots are generated using a frozen policy, whereas the principal baselines do not receive this auxiliary supervision. Reporting 300k control steps for each method therefore does not establish equal total interaction or information budgets. The authors should clarify the cost of obtaining the frozen policy and constructing the auxiliary data, including repeated action prefix replay. More importantly, a baseline trained on the same additional branch trajectories using ordinary prediction objectives would help determine how much improvement comes from the proposed losses.
  2. The central outcome gating mechanism is not isolated by the ablation studies. The comparisons are informative, but the fork component bundles prediction, separation, ranking, and outcome gating. Consequently, they do not establish whether the distinctive contribution (the use of realized physical divergence to decide when separation is appropriate) is necessary. Comparisons against prediction-only auxiliary training, ungated separation, and a fixed-margin alternative would distinguish the proposed mechanism from more conventional auxiliary dynamics. This is particularly important because the paper’s conceptual novelty rests heavily on selective, outcome dependent supervision.
  3. Evidence for better action prediction is limited. Action induced latent distance increases from 0.024 to 0.089, but branch ranking accuracy improves only from 0.64 to 0.66. This leaves the connection between greater latent separation and improved predictive accuracy insufficiently demonstrated.
  4. The gating settings lack sensitivity analysis. The method fixes the threshold at 0.05, intervention magnitude at 0.8, and horizon at 3 steps. Acrobot produces 0 eligible comparisons in the held out representation analysis, raising questions about when the gate supplies useful supervision.
Questions for authors
  1. Could the authors clarify whether the reported 300k control steps include collecting the auxiliary trajectories and training the frozen clean policy? If these costs are additional, reporting them would allow a clearer comparison of the total training budgets.
  2. Do the authors have existing ablations comparing outcome gating with prediction only or ungated supervision using the same auxiliary data? These comparisons would help distinguish the contribution of the gating mechanism from the benefit of additional trajectories.
  3. Could the authors report the fraction of comparisons that passes the gate for each task and explain why none are eligible for Acrobot in the held out analysis? This would clarify how consistently the action fork objectives provide supervision across tasks.
4 comments

Thank you for the careful review and for asking us to account for the extra interaction and isolate what the auxiliary losses contribute.

Full is the submitted four-loss ISI-WM. Returns below are for unseen backgrounds and report the mean across seeds 6–8, with 20 episodes per seed. Where shown, ± denotes the sample SD.

Q1

Could the authors clarify whether the reported 300k control steps include collecting the auxiliary trajectories and training the frozen clean policy? If these costs are additional, reporting them would allow a clearer comparison of the total training budgets.

(Training and data collection budgets)

Response. Thank you for asking us to make the interaction budget explicit. The reported 300,000 control steps cover online training. Training the shared collector on clean inputs takes 30,000 steps, and collecting the auxiliary training trajectories takes another 68,400–264,400 per task. Walker uses 264,400 (9,600 initial prefix, 249,600 prefix replay, 5,200 branch continuation). Charging both one-time costs in full to each seed gives 594,400 control steps, or 1,188,800 simulator steps with action repeat two. At that budget, Full reaches 365.21 ± 65.18 and TD-MPC2, which spends all 594,400 steps online, reaches 194.10 ± 5.72. We account for the 132,200 Walker validation/test collection steps separately (Appendix E, Table 7).

Q2

Do the authors have existing ablations comparing outcome gating with prediction only or ungated supervision using the same auxiliary data? These comparisons would help distinguish the contribution of the gating mechanism from the benefit of additional trajectories.

(Same-data and gate controls)

Response. Thank you for suggesting the same-data ablations. We train ordinary action-conditioned prediction on the same auxiliary trajectories. It reaches 44.80 on Cup and 135.33 on Walker, while Full reaches 527.18 and 365.21. With ranking off and alignment and prediction held fixed, adding gated separation raises return from 136.92 to 676.22 on Cup and from 255.40 to 432.91 on Walker, with gains in all three seeds in both tasks (Appendix E, Table 6 and Table 7). On Walker, making every recorded pair eligible in Full yields 327.31 (Table 8). W2 compares the eligibility rules at a matched margin and selection rate.

Q3

Could the authors report the fraction of comparisons that passes the gate for each task and explain why none are eligible for Acrobot in the held out analysis? This would clarify how consistently the action fork objectives provide supervision across tasks.

(Gate coverage and Acrobot)

Response. Thank you for asking how often the gate supplies supervision. At threshold 0.05, the revised appendix reports these counts across ten held-out roots per task and visual condition, and the mean training pass rates at steps 250,500–300,000 (Table 9).

TaskEligible held-out pair-stepsLate-training pass rate
Acrobot0/1800%
Cartpole178/18089.01%
Cup600/600100%
Finger594/60099.46%
Reacher590/60095.25%
Walker4,124/4,68084.29%

Every Acrobot test pair stays below the standardized gap threshold over steps 1–3, so its gated prediction metrics are undefined. Lowering the threshold to 0.02 admits 18/180 pair-steps in replay (Table 12).

W1

Basically, ISI-WM receives auxiliary trajectories constructed through simulator replay, exact background correspondences, and state derived outcome labels. Its roots are generated using a frozen policy, whereas the principal baselines do not receive this auxiliary supervision. Reporting 300k control steps for each method therefore does not establish equal total interaction or information budgets. The authors should clarify the cost of obtaining the frozen policy and constructing the auxiliary data, including repeated action prefix replay. More importantly, a baseline trained on the same additional branch trajectories using ordinary prediction objectives would help determine how much improvement comes from the proposed losses.

(Interaction budget and auxiliary information)

Response. Thank you for distinguishing interaction from access to privileged supervision. The Walker comparison in Q1 matches simulator steps. The ordinary predictor in Q2 receives the same auxiliary branch trajectories and paired renders, and learns from executed-branch targets. This tests how far ordinary prediction goes with the same trajectories. Exact replay, paired rendering and physical-state labels remain requirements for constructing Full's training supervision (Appendix E, “Training supervision and interaction budget”).

Thank you for asking us to isolate the outcome gate and connect the representation analysis to prediction and planning.

Full is the submitted four-loss ISI-WM. Control returns below are unseen-background means across seeds 6–8, with 20 episodes per seed.

W2

The central outcome gating mechanism is not isolated by the ablation studies. The comparisons are informative, but the fork component bundles prediction, separation, ranking, and outcome gating. Consequently, they do not establish whether the distinctive contribution (the use of realized physical divergence to decide when separation is appropriate) is necessary. Comparisons against prediction-only auxiliary training, ungated separation, and a fixed-margin alternative would distinguish the proposed mechanism from more conventional auxiliary dynamics. This is particularly important because the paper’s conceptual novelty rests heavily on selective, outcome dependent supervision.

(Physical eligibility and margin controls)

Response. Thank you for asking which part of the fork objective drives the gains. Q2 gives the same-data prediction comparison. On Walker, Full reaches 365.21 and making all recorded pairs eligible yields 327.31, with Full higher in each seed. We then fix the margin at 0.09385, the median adaptive margin over eligible training pairs. The physical gate reaches 373.82 and randomizing eligibility within each batch and horizon while preserving the count yields 372.94 (Appendix E, Table 8). These close means leave the benefit of physical selection at a matched rate open. A focused follow-up would repeat the fixed-margin, rate-matched comparison on Cup and Finger at the same three seeds.

W3

Evidence for better action prediction is limited. Action induced latent distance increases from 0.024 to 0.089, but branch ranking accuracy improves only from 0.64 to 0.66. This leaves the connection between greater latent separation and improved predictive accuracy insufficiently demonstrated.

(Predictive accuracy and shared-candidate planning)

Response. Thank you for asking for a direct prediction test. On the same held-out pairs, we count an identification as correct when the prediction is strictly closer to the executed branch than to an eligible alternative. The task-level means show the clearest gains on Cup and Finger, with variation across tasks.

TaskFullTD-MPC2
Cartpole49.63%49.16%
Cup94.19%50.53%
Finger89.20%82.77%
Reacher52.18%61.33%
Walker53.13%51.14%

With the same auxiliary trajectories, prediction only gives 69.72% on Cup and 51.21% on Walker (Table 10). Appendix E.2 reports correct-branch MSE in each model's own latent space. We also let the planners score the same 64 three-step candidates at 100 states per seed, using realized return discounted at 0.99 as the reference. The mean gap to the best candidate is 0.0890 for Full and 0.1220 for TD-MPC2 on Cup, and 0.3444 and 0.3343 on Walker. Candidate returns vary at 76/300 Cup states and 300/300 Walker states (Appendix E, “Ranking ablation and shared-candidate planning”). These tests show where the larger separation accompanies better branch identification or candidate selection.

W4

The gating settings lack sensitivity analysis. The method fixes the threshold at 0.05, intervention magnitude at 0.8, and horizon at 3 steps. Acrobot produces 0 eligible comparisons in the held out representation analysis, raising questions about when the gate supplies useful supervision.

(Threshold, magnitude and horizon sensitivity)

Response. Thank you for asking how the settings affect coverage. In six-task replay, raising the threshold from 0.05 to 0.10 retains 600/600 eligible pair-steps on Cup and changes Walker from 4,124/4,680 to 3,744/4,680. Extending the horizon to five steps at 0.05 gives 0/300 on Acrobot (Table 12). In matched replay, reducing the first-action offset from 0.8 to 0.4 changes Walker eligibility from 79.1% to 70.6%. Separate Walker retraining yields 379.04 at threshold 0.10 and 389.24 with a five-step auxiliary horizon, with a three-step planner throughout (Table 8). For Acrobot, a follow-up would train at 0.02 and 0.05 with the same data and three seeds to test whether the extra eligible pairs improve control.

Thank authors for the detailed rebuttal with additional clarifications and experimental results. In the first response, the matched-budget comparison addresses my concern about the extra training data on Walker. It's convenicing that ISI-WM still outperforms TD-MPC2 when both use 594,400 control steps. The same-data ablations also show that adding gated separation improves returns on Cup and Walker. The per-task results explain why no Acrobot pairs passed the gate and show that branch identification improved much more on Cup than on Walker.

My remaining concern is how the method chooses which pairs to separate. More specifically, on Walker, the physical gate performs better than using all pairs (365.21 vs. 327.31). However, as given in the additional materials, it shows that, with the same margin and number of pairs, physical and random selection give nearly identical returns (373.82 vs. 372.94). I’d appreciate a brief note or discussion or analysis in the main text acknowledging that the benefit of using physical divergence to select pairs is still unclear.

I would maintain my current score : 2 now. Thank authors for the effort again.

Thank you for your follow-up and for recognizing the matched-budget and same-data controls.

My remaining concern is how the method chooses which pairs to separate. More specifically, on Walker, the physical gate performs better than using all pairs (365.21 vs. 327.31). However, as given in the additional materials, it shows that, with the same margin and number of pairs, physical and random selection give nearly identical returns (373.82 vs. 372.94). I’d appreciate a brief note or discussion or analysis in the main text acknowledging that the benefit of using physical divergence to select pairs is still unclear.

(What physical selection adds at a matched margin and eligibility count)

Response. Thank you for suggesting this clarification. We have revised the “Outcome selection” paragraph in Section 4.3. On Walker, Full achieves a higher mean unseen-background return than all-eligible supervision (365.21 vs. 327.31). With the margin and eligibility count matched, physical and random selection yield similar means (373.82 vs. 372.94; Appendix E.2, Table 8). Accordingly, we limit the conclusion from this comparison to the benefit of selective supervision on Walker and describe the physical gate as an implementation based on realized action outcomes.

Reviewer_RPXgOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ISI-WM (Interventional Selective Invariance for World Models), a training framework designed to improve the robustness of visual world models to background variations while preserving action-dependent dynamics. ISI-WM introduces two types of training-time interventions. First, background interventions construct paired observations of the same physical state under different backgrounds and encourage their latent representations to be aligned. Second, action interventions execute alternative actions from a shared initial state and use the resulting physical outcomes to determine whether their latent representations should be separated. The framework incorporates four auxiliary losses, including background alignment, future prediction, latent separation, and branch ranking, to jointly optimize the visual encoder and latent dynamics model. The results show improved average returns over the evaluated baselines on unseen backgrounds, with ablation studies and representation analyses supporting the complementary effects of the two intervention mechanisms. The approach uses privileged simulator information during training while requiring only RGB observations at deployment.

Critical strengths
  • The paper addresses a relevant challenge in visual world models: learning representations that are invariant to irrelevant background variations while preserving action-dependent information. The motivation is clear, and the overall approach is intuitive and easy to follow.

  • The proposed method can be integrated into TD-MPC2 through additional training objectives without modifying its underlying architecture or planning procedure. It introduces no additional trainable parameters and maintains comparable inference latency, which is a practical advantage.

Critical weaknesses
  • My primary concern is the limited novelty of the proposed method. While the intervention-based construction of training pairs provides a different supervision mechanism, the individual objectives are relatively standard, and their combination appears largely incremental. The paper does not sufficiently establish what fundamentally new insight or capability is introduced beyond existing approaches to learning invariant and action-sensitive representations.

  • ISI-WM relies on privileged simulator information to construct matched physical-state observations, execute alternative action branches, and determine whether different actions produce distinguishable outcomes. In contrast, the main baselines do not receive these additional supervision signals or intervention trajectories.

  • The experimental comparison is limited in terms of both baselines and benchmarks. Visual robustness under background and appearance shifts has been extensively studied in prior work, with established benchmarks such as DMControl-GB, DMC-GB2, and the Distracting Control Suite, as well as methods including DrQ/DrQ-v2, SVEA, SODA, and SADA. However, the paper evaluates ISI-WM primarily under a custom dynamic-background protocol without comparisons against these established visual generalization methods.

  • The proposed training pipeline requires precise reconstruction of the same physical state, paired renderings under different backgrounds, and execution of multiple alternative action sequences from an identical initial configuration. While these operations are feasible in simulation, they are difficult to reproduce in real-world robotic settings, where exact state resets, counterfactual action execution, and controlled background replacement are generally unavailable. This substantially limits the applicability of the method beyond instrumented simulators.

  • The paper introduces four auxiliary losses, but the main ablations only compare background-only, action-fork-only, and full variants. In particular, the action-fork component combines prediction, separation, and ranking losses, making it unclear whether the improvements arise from additional dynamics prediction supervision, explicit representation separation, or the proposed outcome-gated ranking objective. Since several of these losses are closely related to existing training objectives, more fine-grained ablations are necessary to establish the contribution and necessity of each component.

  • The experiments are restricted to six relatively simple DeepMind Control Suite tasks, with visual distribution shifts limited to dynamically changing video backgrounds. This setting does not cover other important visual variations, such as lighting, object appearance, camera viewpoint, or occlusion. Moreover, the improvements are not consistent across tasks.

Questions for authors

The proposed prediction loss appears largely equivalent to the latent consistency objective already used in TD-MPC2, except that it operates on additional action-fork trajectories. Could the authors clarify what distinct supervision this loss provides?

Could the authors provide separate ablations for the prediction, separation, and ranking losses to clarify their individual contributions?

3 comments

Thank you for the careful review and for pointing out where the training controls and the contribution needed clarification.

Full denotes the submitted four-loss ISI-WM. Unless specified, returns are based on training seeds 6, 7, and 8, with 20 evaluation episodes per seed and condition.

Q1

The proposed prediction loss appears largely equivalent to the latent consistency objective already used in TD-MPC2, except that it operates on additional action-fork trajectories. Could the authors clarify what distinct supervision this loss provides?

(What the fork trajectories supervise)

Response. Thank you for raising this distinction. We use ordinary TD-MPC2-style stop-gradient latent consistency. The additional supervision consists of observed futures reached by different actions from a shared physical state, with matched renders of each executed branch. We gave the prediction-only control the same auxiliary data, sampling, renders, crops, and 300k online budget. On unseen backgrounds, prediction only scores 44.80 on Cup Catch and 135.33 on Walker Walk, versus 527.18 and 365.21 for Full. The remaining gap shows the value of how Full uses the same trajectories on these two tasks (Appendix E, Table 6 and Table 7).

Q2

Could the authors provide separate ablations for the prediction, separation, and ranking losses to clarify their individual contributions?

(Contributions of prediction, separation, and ranking)

Response. Thank you for requesting the loss-by-loss comparisons. With ranking off, adding alignment and then gated separation to prediction raises unseen returns from 44.80 to 136.92 to 676.22 on Cup and from 135.33 to 255.40 to 432.91 on Walker. Adding separation improves all three seeds on each task. With ranking on, removing separation from Full drops Cup from 527.18 to 151.97. In the six-task rank-off ablation, ranking improves all three Finger Spin seeds and lowers all three Cup and Walker seeds on unseen backgrounds. On held-out Cup forks, ranking raises correct-branch identification from 91.78% to 94.19%. Full also has higher correct-branch MSE when errors are measured in the respective latent spaces (Appendix E, Table 6, Table 7, Table 14, Table 9). These comparisons keep prediction active. To test its necessity in Full, we would run three matched seeds with its auxiliary weight at zero and the other terms and the online loss fixed.

W1

My primary concern is the limited novelty of the proposed method. While the intervention-based construction of training pairs provides a different supervision mechanism, the individual objectives are relatively standard, and their combination appears largely incremental. The paper does not sufficiently establish what fundamentally new insight or capability is introduced beyond existing approaches to learning invariant and action-sensitive representations.

(Contribution of the intervention design)

Response. Thank you for asking us to sharpen the claim. The loss forms are familiar. The contribution is a way to construct and combine supervision with a shared physical reference. Changing only the render gives an alignment target. Changing the action from the same root gives executed futures, whose measured divergence determines when to apply separation and ranking. The matched-data comparisons isolate the extra effect of alignment and gated separation. We describe the state gap as a measure of physical divergence, with a task-dependent relationship to short-horizon reward (Appendix E, Gating, sensitivity and branch prediction).

W2

ISI-WM relies on privileged simulator information to construct matched physical-state observations, execute alternative action branches, and determine whether different actions produce distinguishable outcomes. In contrast, the main baselines do not receive these additional supervision signals or intervention trajectories.

(Simulator supervision and interaction accounting)

Response. Thank you for emphasizing this asymmetry. On Walker, we count 300k online steps, 30k clean collection-policy steps and 264,400 training-root simulator steps, including 249,600 repeated prefix steps. At the same 594,400 control steps, TD-MPC2 uses them all online and scores 194.10 on unseen backgrounds versus 365.21 for Full. This control matches simulator step counts. The prediction-only control in Q1 matches access to the auxiliary trajectories. The outcome labels use simulator state during training, and both deployed policies use RGB observations (Appendix E, Training supervision and interaction budget).

Thank you for raising these experimental and practical questions.

Full denotes the submitted four-loss ISI-WM. We use seeds 6–8 and 20 episodes per seed and condition unless noted.

W3

The experimental comparison is limited in terms of both baselines and benchmarks. Visual robustness under background and appearance shifts has been extensively studied in prior work, with established benchmarks such as DMControl-GB, DMC-GB2, and the Distracting Control Suite, as well as methods including DrQ/DrQ-v2, SVEA, SODA, and SADA. However, the paper evaluates ISI-WM primarily under a custom dynamic-background protocol without comparisons against these established visual generalization methods.

(Coverage of established methods and benchmarks)

Response. Thank you for identifying these comparisons. We used the same 300k online steps and training background split. DrQ-v2, SVEA, and SADA-conv use the shared 64×64 composites resized to 84×84. Below are unseen returns from 20 episodes per seed and condition (mean ± sample SD across seeds, Appendix E, Table 16).

ArmCupWalker
TD-MPC253.18 ± 11.49137.64 ± 8.88
SVEA131.45 ± 28.2333.24 ± 4.83
DrQ-v2498.02 ± 233.2050.14 ± 41.23
SADA-conv155.93 ± 36.50—
Full527.18 ± 73.79365.21 ± 65.18

DrQ-v2 is the closest baseline on Cup. In the separate 50-episode evaluation with matched draws, Full averages 536.66 and DrQ-v2 459.81 (Table 17). Table 18 reports the individual seed results.

On the official DMC-GB2 shift_hard and rotate_hard distributions, frozen Full beats TD-MPC2 for each seed on both tasks using 20 matched episodes per seed (the official runner uses 100, Table 22). W6 gives DCS results. We plan to run SODA and SADA with all augmentations on the same tasks and seeds, and evaluate DMControl-GB appearance shifts with test videos held out from training and auxiliary data.

W4

The proposed training pipeline requires precise reconstruction of the same physical state, paired renderings under different backgrounds, and execution of multiple alternative action sequences from an identical initial configuration. While these operations are feasible in simulation, they are difficult to reproduce in real-world robotic settings, where exact state resets, counterfactual action execution, and controlled background replacement are generally unavailable. This substantially limits the applicability of the method beyond instrumented simulators.

(Sensitivity to reset mismatch and real-robot applicability)

Response. Thank you for highlighting reset precision. Training currently requires an instrumented simulator. In Walker, perturbing one coordinate of the final prefix action by 0.02 or 0.05 flips 66/9,360 or 129/9,360 distinct-branch labels. The gate also fires for 19/780 or 60/780 same-action comparisons from mismatched starts (Table 13). We would test perturbed starts in simulated door opening, then measure reset error and gate disagreement in hardware trials with a fixture and controlled lighting before using those trials for supervision. Deployment uses RGB.

W5

The paper introduces four auxiliary losses, but the main ablations only compare background-only, action-fork-only, and full variants. In particular, the action-fork component combines prediction, separation, and ranking losses, making it unclear whether the improvements arise from additional dynamics prediction supervision, explicit representation separation, or the proposed outcome-gated ranking objective. Since several of these losses are closely related to existing training objectives, more fine-grained ablations are necessary to establish the contribution and necessity of each component.

(Isolating the action-fork loss terms)

Response. Thank you for requesting these controls. On Cup with matched fork data and alignment off, unseen returns are 44.80 for prediction only, 278.98 with separation added, 83.05 with ranking added, and 168.72 with both. With alignment on and ranking off, separation also helps every Cup and Walker seed. Q2 describes the task-dependent effect of ranking (Table 6 and Table 7).

Thank you for asking us to test additional visual shifts and report how their effects vary by task.

Full denotes the submitted four-loss ISI-WM. Unless specified, returns are based on training seeds 6, 7, and 8, with 20 evaluation episodes per seed and condition.

W6

The experiments are restricted to six relatively simple DeepMind Control Suite tasks, with visual distribution shifts limited to dynamically changing video backgrounds. This setting does not cover other important visual variations, such as lighting, object appearance, camera viewpoint, or occlusion. Moreover, the improvements are not consistent across tasks.

(Transfer to additional visual shifts and differences across tasks)

Response. Thank you for recommending the additional shifts. We evaluated frozen 300k checkpoints. With the same evaluation schedule, Full scores above TD-MPC2 at all three seeds on both Cup and Walker for each isolated DCS shift in material color, DAVIS background, and camera (Appendix E, Table 21). With the three shifts combined, Full averages 150.19 across six tasks versus 89.97 for TD-MPC2. TD-MPC2 scores higher on Finger Spin (88.22 versus 73.88), and Acrobot returns are low for both methods (Table 19). We also tested synthetic occlusion and lighting separately across all six tasks.

Shift (six-task means)FullTD-MPC2Full Δ cleanTD-MPC2 Δ cleanFull higher
Synthetic 16×16 occlusion340.97187.33−122.83−72.304/6 tasks
Simulated lighting, intensity 0.482.3570.42−381.45−189.212/6 tasks

The Δ columns measure changes from each policy's matched clean evaluation. Full scores higher in absolute return on four occlusion tasks, while its six-task mean falls more from clean. The 0.4 lighting condition is especially difficult for Full. At intensity 0.7, Full scores 322.05 versus 163.88 on Cup and 186.17 versus 92.89 on Walker. We now qualify the visual robustness claim by shift and task (Table 23). In follow-up manipulation experiments, we would vary camera viewpoint and lighting independently and report task success alongside clean-relative changes.

Reviewer_fKG2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ISI-WM, a training framework for improving the visual robustness of world models. Built upon TD-MPC2, the method introduces two complementary interventions: background alignment to suppress task-irrelevant visual variations, and outcome-gated action forking to preserve action-dependent dynamics. Experiments on six DMC tasks demonstrate improved performance under unseen dynamic backgrounds.

Critical strengths
  1. Clear motivation and intuitive design. The idea of preserving action-relevant dynamics while suppressing background variations is well motivated. The method can be integrated into existing world models without additional trainable parameters.

  2. Promising empirical results. The method achieves substantial improvements over the evaluated baselines, and the ablation experiments suggest complementary benefits from the two interventions.

Critical weaknesses
  1. Limited methodological novelty.

The proposed method mainly combines cross-background feature alignment with action-conditioned prediction, separation, and ranking objectives. These components are closely related to existing invariant representation learning and contrastive learning techniques. Although their integration is reasonable, the methodological novelty beyond existing approaches appears relatively limited.

  1. Questionable definition of task-relevant dynamics.

The proposed outcome gate relies on the distance between simulator states to determine whether different action outcomes should be separated in the latent space. However, physical state differences do not necessarily correspond to task-relevant differences. The paper does not sufficiently justify why this distance metric is an appropriate criterion for learning control-relevant representations.

  1. Limited experimental scope.

The evaluation is restricted to relatively simple DMC tasks with synthetically replaced 2D backgrounds. It remains unclear whether the method generalizes to more complex robotic manipulation tasks or realistic visual distribution shifts.

  1. Minor Comments.

Figure 2 has noticeable presentation issues. Several row labels, including "Seen", "Unseen", and "Background Video Frame", overlap with the image panels and are partially obscured. The font sizes are also too small, making the figure difficult to read.

Questions for authors

Q1. Real-world robotic applicability.

The experiments are limited to relatively simple DMC control tasks. Could the authors evaluate ISI-WM on more challenging robotic manipulation benchmarks, such as Meta-World, ManiSkill, or RoboSuite, involving tasks like pick-and-place, drawer opening, or object insertion? A real-world robotic evaluation would further strengthen the practical relevance of the proposed method.

Q2. Downstream applications.

The intended downstream applications of ISI-WM are not sufficiently clear to me. Beyond the current DMC control tasks, what practical robotic applications would benefit from this framework? Could the authors clarify its potential use cases and explain how the evaluated tasks relate to realistic robotic control problems?

Q3. Dependence on simulator supervision.

The method relies on simulator states, matched background renderings, and action-fork trajectories during training. How could these interventions be constructed in real-world robotic settings where exact state reconstruction and counterfactual action execution are difficult?

Q4. Limited visual distribution shifts.

The experiments primarily consider 2D background replacement while keeping the foreground objects and physical dynamics unchanged. Would the method remain effective under more realistic visual changes, such as camera viewpoint shifts, lighting variations, object appearance changes, or partial occlusions?

Q5. Effectiveness of action-fork supervision.

Could the authors provide separate ablations for the prediction, separation, and ranking losses? Additionally, Figure 5 shows that action-branch ranking accuracy does not improve over TD-MPC2 despite increased latent separation. How do the authors explain this result?

3 comments

Thank you for the careful review and for encouraging us to clarify how the training setup might extend to robotics.

Q1

Q1. Real-world robotic applicability.

The experiments are limited to relatively simple DMC control tasks. Could the authors evaluate ISI-WM on more challenging robotic manipulation benchmarks, such as Meta-World, ManiSkill, or RoboSuite, involving tasks like pick-and-place, drawer opening, or object insertion? A real-world robotic evaluation would further strengthen the practical relevance of the proposed method.

(Robotic manipulation and hardware evaluation)

Response. Thank you for suggesting this extension. Our current evidence covers six simulated DeepMind Control tasks. We would first test pick-and-place and drawer opening in ManiSkill, where saved pre-contact states allow repeatable comparisons between action sequences. Using the same RGB observation protocol and matched interaction budgets, we would compare TD-MPC2, same-data ordinary prediction, and the submitted four-loss ISI-WM (Full) on task success under held-out lighting and object appearances. A hardware follow-up would start with a tabletop pick-and-place task and use repeatable object placement plus an external grasp-success measurement. These are proposed evaluations.

Q2

Q2. Downstream applications.

The intended downstream applications of ISI-WM are not sufficiently clear to me. Beyond the current DMC control tasks, what practical robotic applications would benefit from this framework? Could the authors clarify its potential use cases and explain how the evaluated tasks relate to realistic robotic control problems?

(Downstream tasks and what the current benchmarks probe)

Response. Thank you for asking us to make the connection explicit. Cup Catch probes timing and interception from RGB. Walker Walk probes feedback control of a moving body. A potential application is a camera-guided gripper picking objects from a conveyor. A change in lighting should leave the object's perceived position stable, while two different grasp timings can produce different outcomes that the planner needs to distinguish. The benchmark tasks test these control demands in isolation. Conveyor picking would test their interaction with contact and grasping.

Q3

Q3. Dependence on simulator supervision.

The method relies on simulator states, matched background renderings, and action-fork trajectories during training. How could these interventions be constructed in real-world robotic settings where exact state reconstruction and counterfactual action execution are difficult?

(Approximate visual pairing and action branches on hardware)

Response. Thank you for raising the issue of reset precision. A hardware pilot could hold the robot and object in place while changing lighting or the background to collect matched images. For action branches, we would reset with fixtures, match the measured starting pose using joint readings and external object tracking, and execute separate trials. We would log residual pose error, discard poorly matched starts, and compare gate labels and task success across reset tolerances. These measurements would be training-time supervision. Deployment uses RGB.

We took a first step with a simulated reset-mismatch probe (Table 13). On 10 held-out Walker roots, offsetting one coordinate of the last replay-prefix action by magnitudes of 0.02 and 0.05 changed 66 and 129 of 9,360 distinct-branch gate decisions, respectively. Each decision tests a pair of outcomes at a given prediction step. More than 98.6% of the decisions agreed with the exact replay at each tested magnitude.

Thank you for asking us to test a wider range of shifts and to separate the effects of the auxiliary losses. Full denotes the submitted four-loss ISI-WM. Returns below are means across seeds 6–8, with 20 episodes per seed and condition.

Q4

Q4. Limited visual distribution shifts.

The experiments primarily consider 2D background replacement while keeping the foreground objects and physical dynamics unchanged. Would the method remain effective under more realistic visual changes, such as camera viewpoint shifts, lighting variations, object appearance changes, or partial occlusions?

(Visual robustness beyond video backgrounds)

Response. Thank you for suggesting these tests. We evaluated frozen checkpoints under isolated dynamic camera, material-color and video-background shifts (Table 21). Full scored higher than TD-MPC2 for all three training seeds on Cup Catch and Walker Walk under each shift. With combined DCS perturbations, the six-task mean was 150.19 for Full and 89.97 for TD-MPC2 (Table 19).

We also tested six tasks under dim lighting and synthetic partial occlusion. The equally weighted task means and changes from each method's clean reference are below (Table 23).

ShiftFullTD-MPC2Full changeTD-MPC2 change
16×16 occlusion340.97187.33−122.83−72.30
Lighting scale 0.482.3570.42−381.45−189.21

Full had higher absolute return on four tasks under occlusion and two under dim lighting. The clean-relative changes show that strong dimming remains challenging.

Q5

Q5. Effectiveness of action-fork supervision.

Could the authors provide separate ablations for the prediction, separation, and ranking losses? Additionally, Figure 5 shows that action-branch ranking accuracy does not improve over TD-MPC2 despite increased latent separation. How do the authors explain this result?

(Contribution of each fork loss and the interpretation of Figure 5)

Response. Thank you for asking us to separate the losses and clarify Figure 5. With fixed auxiliary data and alignment off, Cup's unseen return was 44.80 for ordinary prediction, 278.98 with gated separation added, and 83.05 with ranking added. With alignment on and ranking off, gated separation raised Cup from 136.92 to 676.22 and Walker from 255.40 to 432.91, with gains in all three seeds on both tasks (Table 6 and Table 7). With alignment and ranking both on, the Cup return was 151.97 without separation and 527.18 with it. The six-task rank-off comparison holds the other losses fixed. On unseen backgrounds, ranking helps Finger, whereas Cup and Walker score higher with ranking off in all three seeds (Table 15). Prediction serves as the shared anchor in these controls. To isolate its contribution within Full, the next control is three 300k-step runs with only auxiliary prediction disabled, keeping TD-MPC2's online loss, auxiliary data and other three terms fixed.

Figure 5 uses equal task weights. The five-task identification means are 58.99% for TD-MPC2 and 67.66% for Full. Identification depends on the prediction's distance to each target. In the matched held-out probe, accuracy for TD-MPC2, ordinary prediction and Full was 50.53%, 69.72% and 94.19% on Cup, and 51.14%, 51.21% and 53.13% on Walker (Table 10). Figure 5 and Appendix E.2 state the aggregation and distinguish geometric distance, correct-target error and identification.

W1

  1. Limited methodological novelty.

The proposed method mainly combines cross-background feature alignment with action-conditioned prediction, separation, and ranking objectives. These components are closely related to existing invariant representation learning and contrastive learning techniques. Although their integration is reasonable, the methodological novelty beyond existing approaches appears relatively limited.

(Novelty of the intervention-based supervision)

Response. Thank you for asking us to sharpen the contribution. We use familiar objectives with a pairing rule based on what was changed and what actually happened. Re-rendering one state specifies differences to align, while branching from a shared state lets the realized outcome determine which pairs receive separation. The fixed-data comparisons in Q5 show what this adds beyond fitting ordinary predictions to the same executions. Section 3 describes how we construct the pairs and gate labels.

Thank you for the feedback on the gate, the scope of evaluation and the presentation.

W2

  1. Questionable definition of task-relevant dynamics.

The proposed outcome gate relies on the distance between simulator states to determine whether different action outcomes should be separated in the latent space. However, physical state differences do not necessarily correspond to task-relevant differences. The paper does not sufficiently justify why this distance metric is an appropriate criterion for learning control-relevant representations.

(What physical divergence tells us about control relevance)

Response. Thank you for raising this distinction. We use standardized physical distance to identify when two actions taken from the same state produce distinguishable outcomes. To check when this criterion tracks control consequences, we replayed the held-out actions and measured realized 1–3-step returns. On Walker, gate-eligible pairs had larger mean absolute return gaps in all nine strata matched by first-action distance and horizon. On Cup, 568 of 600 eligible pair-steps had tied short-horizon returns, even though the physical outcomes diverged. Appendix E, “Gating, sensitivity and branch prediction,” describes this task dependence.

On Walker, the submitted four-loss model (Full) scored 365.21 on unseen backgrounds versus 327.31 when all pairs were eligible, with higher returns in each seed. At a fixed margin, the physical gate and a randomized gate with the same eligibility count scored 373.82 and 372.94, respectively (Table 8). These controls support selective supervision over the all-eligible setup in this comparison. Longer-horizon success or progress labels would provide a direct test of alternative gating criteria where near-term rewards are sparse.

W3

  1. Limited experimental scope.

The evaluation is restricted to relatively simple DMC tasks with synthetically replaced 2D backgrounds. It remains unclear whether the method generalizes to more complex robotic manipulation tasks or realistic visual distribution shifts.

(Benchmark coverage and generalization to manipulation)

Response. Thank you for asking us to test the scope of generalization. In addition to the isolated shifts in Q4, we evaluated frozen checkpoints on the official DMC-GB2 shift_hard and rotate_hard distributions using our shared 20-episode protocol. Full scored higher than TD-MPC2 in all three matched seeds on both tasks. Under rotate_hard, the means were 287.10 vs. 83.98 on Cup and 103.31 vs. 51.88 on Walker (Table 22). This broadens the evidence on simulated visual shifts. The ManiSkill evaluation in Q1 would test whether the findings extend to contact-rich manipulation and held-out appearances.

W4

  1. Minor Comments.

Figure 2 has noticeable presentation issues. Several row labels, including "Seen", "Unseen", and "Background Video Frame", overlap with the image panels and are partially obscured. The font sizes are also too small, making the figure difficult to read.

(Figure 2 label placement and legibility)

Response. Thank you for identifying the overlap. In revised Figure 2, the row labels are larger and sit in a dedicated column outside the image panels.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.