acceptodds
Under review as a conference paper at ICLR 2027

Causal Nuisance Interventions for Distractor-Robust World Models

Abstract

World models trained from pixels can remain sensitive to camera motion and changing backgrounds, even when trained without pixel reconstruction. We introduce Causal Nuisance Interventions (CaNI), a causal formulation and replay-training method using a specified nuisance structural causal model (SCM) to guide observation-proxy design before training. We instantiate CaNI on a decoder-free Dreamer backbone by applying one randomly sampled padded translation consistently across each replay sequence, while retaining the architecture and learning objectives. We prove that uniform posterior stability is sufficient to bound changes in a fixed recurrent policy's return under observation interventions that preserve the controlled process. Across five DeepMind Control tasks with driving-video backgrounds and camera jitter, CaNI achieves higher mean return than each of five evaluated baselines on every task, increasing task-averaged return from 310.5 for the matched backbone to 715.5 for CaNI (approximately 130%). On the driving-video benchmark without jitter, time-averaged return improves by 15% with comparable final returns. Frozen-policy evaluations show improved robustness to evaluation-time jitter, and disjoint training and evaluation splits of the DAVIS video dataset show better performance on unseen backgrounds on two tasks. Posterior diagnostics and controlled ablations support reduced shift sensitivity and the intervention design.

Primary area:
reinforcement learning
Keywords:
model-based reinforcement learning, world models, causal interventions

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PBYhOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper argues that world models trained on pixels without reconstruction still fail when the camera moves or the background changes. The remedy is a change to the training data rather than to the model. A causal model states which image factors can change, background and camera placement, and which must not, the task physics. Training then shifts each recorded sequence by one fixed offset, keeps actions, rewards and labels as recorded, and leaves the network, its objective and its inference untouched.

The analysis shows that if the model's hidden state barely moves when the image shifts, a fixed policy's score can move only a little. With jitter in both training and testing, the five-task average score on DMC is 715.5 (on the usual 0 to 1000 scale) against 310.5 for the same network trained without the intervention. When jitter appears only at test time, the baseline falls from 863 to 314 while the method falls from 849 to 650, so the gain lies in holding up under unseen image change rather than in raising the ceiling.

Critical strengths
  1. The paper contributes a training-time data intervention guided by a causal model, and shows that it improves control under camera jitter. Fixing background and camera placement as the only factors allowed to vary reduces the intervention to one translation offset per sequence. It needs no architectural change and nothing at test time.

  2. The paper locates the source of robustness in the training data rather than in added capacity. The intervention changes nothing about the architecture, the objective or the inference, so the gain on a matched backbone cannot be credited to extra parameters or a new training objective, and a policy trained without jitter degrades far less than the baseline when jitter is added at test time. The stability argument relates the return change to how far the latent state moves under shift, which supports that reading.

Critical weaknesses
  1. The claims outrun the evidence. The abstract and introduction speak about decoder-free world models as a class, while every experiment uses a single R2-Dreamer, five DMC tasks and three seeds, with camera jitter implemented as a 2D image translation over DAVIS backgrounds. No result swaps in another decoder-free backbone, and no nuisance other than translation is tested. The practical claim that the causal model tells you which proxy to choose is therefore demonstrated on one configuration only.

  2. The theory is conditional and its premise is never instantiated. The bound requires posterior movement to be uniformly limited across all reachable states and intervention pairs, whereas Section 4.4 gives a Jensen-Shannon estimate over 81 pad-4 translations, a reduction of about a quarter, which is a sanity check rather than an instantiation. The causal argument itself is only supported ordinally, since in Table 1 brightness and contrast also improves all three tasks.

Questions for authors

The experiments cover one decoder-free backbone and five DMC tasks, while the abstract states the claim more broadly. Could the authors clarify the intended scope of the conclusions and which aspects of generalization remain untested?

Reviewer_jP5fOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

CaNI applies one padded random translation to every frame in a replay sequence while keeping R2-Dreamer's architecture and training objectives unchanged. A manually specified SCM motivates this intervention, and the theory bounds changes in a fixed recurrent policy's return under a uniform posterior-stability assumption. On five DMC tasks with video backgrounds and image-plane jitter, task-averaged return rises from 310.5 to 715.5. The experiments also test frozen policies under evaluation-time jitter and transfer to unseen DAVIS backgrounds on two tasks.

Critical strengths
  1. The matched comparison gives a clear and substantial result. CaNI keeps R2-Dreamer's architecture, objectives, and optimization settings, changing how replay observations are transformed. It improves the mean return on all five tasks in Table 5, by about 405 points on average. I find the frozen-policy test especially useful: policies trained without benchmark jitter score about 650 for CaNI versus 314 for R2-Dreamer when jitter is introduced at evaluation (Section 4.3). This is direct evidence of improved robustness with the policy parameters fixed.

  2. The additional tests address relevant questions about the gain. The disjoint DAVIS split tests unseen background videos, with improvements on both tasks with and without jitter. The temporal-coupling and augmentation-family ablations examine concrete design choices. The authors are also careful about the theory's scope: the posterior diagnostics are sampled sensitivity measurements, exact abstraction needs control sufficiency, and reward-sensor noise is an empirical extension beyond the observation-only theory.

Critical weaknesses
  1. It is unclear how much the SCM adds to the training method. Sequence-consistent shifts before RSSM inference already appear in DreamerPro [1], as the paper acknowledges. The SCM is specified by hand and leads to the same fixed translation rule throughout, with no new training objective or adaptive intervention selection. The brightness/contrast comparison shows that translation suits this displacement benchmark, but gives little evidence that the SCM helps choose interventions more generally. I see the main contribution as the controlled finding on this backbone and nuisance family.

  2. The link between the theorem and the trained model is incomplete. Theorem 1 assumes uniform posterior-TV stability. Training averages the original agent objective over translations, and the paper does not show that this optimization yields the required stability. Appendix D.1 reports that the JS-derived TV bound saturates at one. The measured decrease in sensitivity is informative, but it does not supply a nontrivial guarantee or establish why returns improve.

  3. Information loss near image boundaries is not measured. Section 3.1 requires task information to remain observable or recoverable after transformation. The radius study in Section 4.5 covers Cheetah and Walker, while Section 6 specifically notes that translation can crop the ball in Cup Catch. The average returns do not resolve whether the needed information remains recoverable in those boundary cases.

  4. The generalization tests remain narrow. Section 4.1 uses one backbone and five DMC tasks, with camera jitter implemented as post-render image translation. Section 4.3 and Figure 6 show useful transfer to unseen DAVIS videos on Cheetah and Walker. The evidence supports new backgrounds on those tasks, but leaves transfer to other control domains and camera transformations open.

[1] DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations. ICML 2022.

Questions for authors
  1. What concrete design choice does the SCM add beyond selecting a label-preserving, sequence-consistent augmentation for the known camera mechanism? Can it guide a useful choice outside the translation setting studied here?

  2. For no augmentation, independent per-frame shifts, and sequence-shared shifts, does lower posterior sensitivity accompany higher return on the same tasks? Reporting both measurements for these configurations would help assess the proposed explanation. If their rankings disagree, the sensitivity result would be better treated as a separate diagnostic.

  3. Table 5 reports 935.5 ± 35.1 for CaNI versus 86.6 ± 1.3 for R2-Dreamer on Cup Catch, but Table 8 tests radii 2, 4, and 8 only on Cheetah and Walker. Given the risk of cropping the ball, could you extend this comparison to Cup Catch? Please report variation across training seeds and how often augmentation partially or fully crops a previously visible ball, to assess whether the gain holds as more visual information is at risk.

Reviewer_ix63OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the robustness of pixel-based recurrent world models to visual nuisances, particularly moving backgrounds and camera jitter. The authors propose Causal Nuisance Interventions (CaNI), a framework that uses a specified nuisance structural causal model to motivate observation transformations that preserve the underlying controlled transition, reward, and continuation process. The concrete instantiation applies a randomly sampled, sequence-consistent padded translation to replay observations before training an otherwise unchanged R2-Dreamer agent. The paper also establishes a return-difference bound for fixed recurrent policies under uniform posterior total-variation stability. Experiments on five DeepMind Control tasks show substantial improvements over several baselines under video backgrounds and camera jitter. Additional frozen-policy evaluations, held-out DAVIS background experiments, posterior sensitivity measurements, and augmentation ablations provide further empirical evidence.

Critical strengths
  1. Strong empirical improvements with a well-controlled backbone comparison. The most compelling contribution is the substantial improvement over the matched R2-Dreamer backbone. In the primary benchmark, CaNI improves the five-task average return from 310.5 to 715.5 and achieves higher mean performance on all five tasks. Since the architecture and training objectives are preserved, the comparison provides relatively clean evidence for the utility of the proposed replay intervention. The frozen-policy evaluation is particularly valuable. The improvement under newly introduced evaluation-time jitter demonstrates robustness beyond simply comparing independently trained policies.

  2. Useful ablations and representation-level diagnostics. The comparison between per-frame and sequence-shared shifts supports the importance of temporal consistency for recurrent world models. The posterior sensitivity analysis also shows reduced sensitivity to the tested translation family. The held-out DAVIS experiments offer additional evidence that the benefits can extend beyond the backgrounds seen during training. Together, these experiments make the empirical findings more convincing than a standard benchmark comparison alone.

Critical weaknesses
  1. The methodological novelty relative to existing augmentation techniques is insufficiently established. The implemented method amounts to adding sequence-consistent random spatial translations to the existing R2-Dreamer training pipeline, without changing the architecture or objectives. However, temporally consistent random shifts have already been incorporated into recurrent world-model training, notably in DreamerPro (Deng et al., ICML 2022). Although the learning objectives and backbone differ, the central augmentation mechanism is not new. The proposed SCM provides a plausible interpretation of why translation is appropriate for camera displacement. Nevertheless, it remains unclear what operational capability the causal formulation contributes beyond selecting an intuitive, previously established augmentation. The brightness/contrast comparison partially supports the importance of intervention choice, but it does not establish that SCM guidance offers a distinctive advantage over conventional mechanism-matched augmentation design. This limits my confidence in the claimed methodological contribution.

  2. The theoretical robustness condition is not directly supported by the empirical measurements or guaranteed by the training objective. Theorem 1 establishes a sufficient condition for return stability under a uniform total-variation bound on posterior distributions. The proof appears reasonable under its stated assumptions. However, the proposed training objective does not explicitly enforce this condition, and the experiments evaluate average groupwise Jensen–Shannon sensitivity over sampled histories and a finite translation set. Importantly, Appendix D.1 acknowledges that converting the observed sensitivity values into the corresponding TV upper bounds yields a vacuous bound of one. Consequently, the current diagnostics do not provide a nontrivial quantitative connection to the return guarantee. Furthermore, the evaluation-time jitter includes temporally varying subpixel shifts and different boundary behavior from the training proxy. The theoretical analysis therefore does not directly certify robustness under the evaluated camera process. While I appreciate the authors' acknowledgment of these limitations, the theoretical contribution currently provides a conditional motivation rather than an explanation of the learned method's robustness.

Questions for authors

Q1. What is the distinctive operational contribution of the SCM beyond motivating a standard sequence-consistent translation augmentation? Can the authors identify a concrete prediction or design decision that would differ from a conventional, carefully tuned augmentation approach? A targeted example involving another nuisance mechanism would help establish the generality and necessity of the causal formulation.

Q2. Can the authors clarify whether any nonvacuous relationship between the measured posterior sensitivity and the theoretical return bound can be established for the evaluated agents? Alternatively, could the theoretical claims be more explicitly separated from the empirical robustness results, given that the training objective does not guarantee uniform posterior stability?

Q3. How much of the observed robustness extends beyond post-render image-plane translation, such as to alternative camera geometries or visual changes not approximated well by a spatial shift? A targeted evaluation or clarification would help determine whether CaNI should be understood as a general nuisance-intervention framework or primarily as an effective training intervention for translation-based camera perturbations.

Comments on the paper

1 comment
Area_Chair_UQtJOpenReview ↗

Reminder: Please Review the Authors' Rebuttal

Dear Reviewers,

As the discussion phase is approaching its end, I would like to kindly remind you to review the authors' rebuttal and provide your feedback if you have not done it. Please let us know whether the authors' responses address your concerns and update your assessment if appropriate. Your engagement in the discussion is important to ensure a fair and thorough review process.

Thank you for your time and effort!

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.