acceptodds
Under review as a conference paper at ICLR 2027

Learning Successor-Measure World Models for Zero-Shot Control from Pixels

Abstract

World models typically support decision-making by predicting future observations or latent states, with behavior adapted to a task through planning or policy optimization. For zero-shot control, we instead ask whether a world model can directly organize the long-horizon visitation induced by different behaviors, so that a new reward can select among them without additional policy optimization. We propose SWM, a successor-measure world model for zero-shot visual control. Rather than learning step-by-step predictive dynamics, SWM learns policy-conditioned future occupancy in a compact pretrained visual latent space. SWM uses a frozen DINO encoder as a task-agnostic visual basis and trains a lightweight adaptor, together with forward and backward successor-measure representations, to organize visual observations into a space where future occupancy can be factorized and reused across tasks. At test time, a new task specified by reward-labeled observations is projected onto the learned backward basis, directly selecting a policy from the pretrained behavior family without additional policy optimization or online interaction. Experiments on pixel-based Control tasks show that SWM consistently improves over reward-free predictive world-model baselines and achieves higher mean return than CNN-FB on all sixteen evaluated tasks. Importantly, the learned latent space preserves control-relevant pose information and retains robustness under visual perturbations, suggesting that pretrained visual features can serve as a reusable basis for occupancy-based world modeling when aligned through successor-measure factorization.

Primary area:
reinforcement learning
Keywords:
world model, reinforcement learning, representation learning, zero shot

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_aiEuOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Successor-measure World Model (SWM) for zero-shot visual control. The central idea is to combine a frozen DINOv2 visual encoder with the Forward-Backward (FB) successor-measure framework: three consecutive DINO CLS features are concatenated and passed through a lightweight LayerNorm + linear projector, after which the projected representation is used to learn policy-conditioned successor measures and a latent-conditioned policy family. At test time, reward-labeled observations are projected through the learned backward representation to obtain a task vector, which directly selects a pretrained policy without downstream policy optimization, planning, or additional environment interaction.

Critical strengths
  1. The empirical result is consistent and the core design motivation is reasonable The most convincing result is Table 1. SWM outperforms CNN-FB on all 16 downstream tasks, including both relatively static tasks such as Walker Stand and dynamically demanding tasks such as Cheetah Run. The domain-average gains over CNN-FB are reported as 20.2% on Walker, 17.2% on Quadruped, 47.7% on Cheetah, and 30.0% on Jaco. This provides reasonably strong evidence that frozen DINO features can be successfully integrated with FB-style zero-shot control.
  2. The paper goes beyond reporting a performance gain The robustness experiment and representation probing are useful attempts to understand why the method works rather than simply reporting higher returns. In particular, Figure 3 shows that SWM-CLS does not uniformly improve generic pose or motion decodability over SWM-raw, while the control performance gap between the two representations is enormous. This is a meaningful observation: representation quality for generic state prediction and representation quality for successor-measure control need not be equivalent.
Critical weaknesses
  1. Limited methodological novelty. SWM retains the established FB factorization, training objective, latent-conditioned policy, and reward-projection inference. Its concrete addition consists of frozen DINO features, three-frame concatenation, and a LayerNorm–linear projector. The distinction between future occupancy and predictive dynamics therefore largely reflects existing successor-measure learning rather than a new modeling principle. Substantial space is devoted to preliminaries and reformulations of this inherited machinery, while the methodological advance remains limited. The results demonstrate a useful visual implementation, but do not sufficiently establish the conceptual contribution suggested by the world-model framing.

  2. Confounded baseline comparisons. The experiments do not adequately separate the benefit of occupancy modeling from that of pretrained visual representations. SWM uses frozen DINO features, whereas Dreamer and TD-MPC2 are adapted to reward-free pretraining and downstream planning with frozen dynamics. Matching interaction budgets and reward labels supports a comparison of these complete systems, but does not isolate the proposed modeling advantage. A predictive baseline using the same pretrained visual features and comparable adaptation is needed to substantiate the motivation for replacing predictive world modeling with successor-measure modeling.

  3. Incomplete mechanism analysis. The projector ablation demonstrates its importance, but removing it simultaneously changes feature normalization, dimensionality, conditioning, and the availability of a learned shared transformation. It therefore does not establish that the improvement specifically arises from better occupancy factorization. The state probes show that generic decodability does not explain control performance, but do not identify what does. Normalization-only controls, fixed projections at matched dimensionality, and alternative projector objectives would help distinguish the claimed representation benefit from ordinary feature-conditioning and optimization effects.

  4. Limited zero-shot validation. The sixteen tasks demonstrate useful performance within four simulator domains, but the experiments do not establish how reliably the reward-to-policy interface operates under different inference conditions. In particular, sensitivity to the reward-label budget, reward sparsity, and coverage of reward-bearing states is not evaluated. Predicted future occupancy and reward-projection accuracy are also not directly assessed against observed trajectories. Consequently, the evaluation supports improved returns on the selected tasks, while providing limited evidence about the reliability and limitations of the proposed reusable world-model representation.

Questions for authors

See the weaknesses.

3 comments

We thank the reviewer for recognizing the consistent control improvements and the value of the robustness and representation analyses. The additional experiments strengthen the design argument, clarify the role of the visual prior and its adaptation, and examine the label budget needed for downstream task specification. Shared results are provided in C1–C4 of our common response.

1. Contribution and world-model framing — W1

SWM contributes a methodological extension of reward-free FB that replaces visual-encoder training from environment data with fixed pretrained visual knowledge. Our controls identify the sources and scope of its gains: the random-ViT control supports the value of visual pretraining, while competitive R3M results show that SWM is not tied to DINO (C1); fixed projection and projector training make distinct contributions (C2); and the same frozen features and adaptor improve FB in every domain but degrade TD-MPC2 on all locomotion domains, suggesting that the benefit is specific to successor-measure learning (C4). This design achieves higher mean returns than CNN84 in all four domains using approximately 43% of its target-domain FLOPs (C3). We will foreground this method extension and its new empirical findings, condense the FB preliminaries, and clearly attribute the inherited factorization and inference machinery.

2. Predictive comparisons with shared visual features — W2

C4 compares TD-MPC2 and SWM with the same frozen DINO encoder and projector architecture, matching the visual prior while contrasting predictive planning with successor-measure control.

With the same frozen DINO features and adaptor, SWM beats CNN-FB in all four domains (C3), whereas TD-MPC2 beats its CNN version only on Jaco and falls below it on all three locomotion domains (C4). DINO-based TD-MPC2 also stays below SWM in all four domains. This contrasting pattern shows that adding the same visual front end does not consistently improve control, suggesting that how these features are adapted and used for control matters. Together with the adaptor ablations in C2, these results support SWM’s design.

The DINO WM CLS baseline (goal image needed, so only in Jaco) provides a complementary shared-feature comparison, where SWM also outperforms predictive planning on Jaco (64.19 vs 47.6).

Together, the comparisons with DINO WM CLS planning and DINO-based TD-MPC2 show that SWM achieves higher mean returns than predictive alternatives using the same pretrained visual features, supporting its successor-measure world modeling design.

3. What the adaptor controls establish — W3

C2 distinguishes the effects of fixed feature transformation and projector training. LN alone helps Walker and Quadruped , not Cheetah or Jaco. A frozen random projection to the same dimension, followed by LN, reaches 59% / 87% / 87% / 31% of full SWM on Cheetah / Walker / Quadruped / Jaco with no learning in the projector. These gains are consistent with the feature-conditioning explanation raised by the reviewer. Training the projection with the FB loss, at the same dimension and with LN, adds a further 69% / 16% / 15% / 220%. We will revise 5.3 to distinguish the gains from fixed feature transformations from the additional gains obtained by training the projector. Fixed transformations recover a large fraction of locomotion performance, while training yields the largest relative improvements on Cheetah and Jaco.

PCA, a fixed projection that preserves visual-feature variance, performs no better than the random projection. Full-SWM improves over both PCA compression and random fixed projections. Together, these results validate the complete adaptor and establish FB-driven projector learning as a source of control gains beyond fixed feature transformations.

4. Reward-label budget and zero-shot task specification — W4

With all networks frozen and the inference rule unchanged, SWM retains 89–97% of the default mean return on locomotion and 72% on Jaco using 256 reward labels—one twentieth of the 5,120-label default.

The table reports mean normalized returns over four tasks per domain, with N = 5,120 as the 100% reference [95% CI].

DomainN = 256N = 1,024N = 5,120N = 20,480
Walker89% [75, 106]94% [77, 116]100%97% [85, 114]
Quadruped97% [89, 107]97% [89, 108]100%99% [92, 108]
Cheetah89% [70, 121]110% [93, 141]100%111% [94, 139]
Jaco72% [48, 132]83% [56, 159]100%96% [68, 173]

These results support substantially reduced-label task specification for the evaluated models, with greater low-budget degradation and uncertainty on sparse-reward Jaco. Increasing the budget beyond the default yields no consistent mean-return improvement across domains.

AuthorsOpenReview ↗

Follow Up to your Review

Dear Reviewer aiEu,

Thank you again for the time and effort you put into reviewing our work!

We understand that this might be a super busy period for you. We wanted to briefly let you know that we have posted point-by-point responses to your questions and concerns.

As the rebuttal period is ending very shortly, we would be grateful for your feedback on whether the responses address your concerns. We would be happy to clarify anything that remains open.

Thank you!

Best,

The Authors of Submission 47327

AuthorsOpenReview ↗

A Quick Summary of New Results for W1–W4 & Follow-Up

Dear Reviewer aiEu,

As the discussion period is closing soon, here is a short summary of how our new results (C1–C4) address your concerns:

  • W1 – Reframing & broader empirical scope: We will condense the FB preliminaries and foreground our methodological extension alongside new empirical findings across encoders (R3M, CLIP, random ViT in C1) and standard resolution (CNN84 in C3, where SWM still wins with 2.34x fewer FLOPs).
  • W2 – The gain comes from successor-measure modeling, not DINO alone: With the same frozen DINOv2 features and adaptor architecture, SWM outperforms TD-MPC2 in all four domains (DINO even hurts TD-MPC2 on locomotion) and beats DINO-WM on Jaco (64.19 vs. 47.60) (C4).
  • W3 – FB learning adds gains beyond feature conditioning: At matched dimension with LN, the FB-trained projector improves over a frozen random projection by +69% / +16% / +15% / +220% on Cheetah / Walker / Quadruped / Jaco, while variance-preserving PCA is no better than random (C2).
  • W4 – Task specification is label-efficient: With 1/20 of the default reward labels (256 labels), SWM retains 89–97% of its return on locomotion and 72% on Jaco.

We believe these results address the main concerns in your review, and we would be grateful if you would consider updating your assessment.

Best regards, The Authors of Submission 47327

Reviewer_UGDnOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes the Successor-Measure World Model (SWM), which integrates a frozen DINOv2 visual encoder, short-term temporal feature concatenation, and a lightweight trainable projector into the existing forward–backward (FB) successor-measure learning framework. During reward-free pretraining, SWM jointly learns representations of policy-conditioned future occupancy and a family of latent-conditioned policies. At test time, the model projects reward-labeled observations onto the learned backward representation to obtain a task vector, which directly selects a pretrained policy without additional policy optimization or test-time planning.

Critical strengths

SWM achieves higher mean zero-shot returns than CNN-FB on all 16 evaluated tasks, covering both locomotion and robotic manipulation. These results demonstrate the effectiveness of combining pretrained visual representations with successor-measure learning and suggest practical potential for efficient zero-shot visual control.

Critical weaknesses
  1. Limited methodological novelty and insufficient isolation of the projection mechanism. The main methodological contribution is the integration of frozen DINOv2 features, short-term temporal concatenation, and a lightweight LayerNorm–Linear projector into the existing FB framework. However, the projector ablation simultaneously removes both LayerNorm and the linear projection, making it difficult to distinguish the individual effects of feature normalization, dimensionality reduction, and FB-objective-driven representation adaptation. Consequently, the substantial performance gap between SWM-raw and SWM-CLS does not sufficiently establish whether the improvements specifically arise from learning a control-relevant representation through the FB objective or from simpler feature transformations.
  2. Insufficiently controlled baseline comparisons and computational efficiency evaluation. Dreamer-v3 and TD-MPC2 are substantially adapted to the reward-free pretraining and zero-shot control setting. In particular, their downstream CEM planning uses a horizon of 15, without terminal value estimation or policy-prior trajectories. These choices may limit the performance of predictive world models, making it difficult to determine whether the reported comparisons reflect their capabilities under appropriately optimized adaptations. Furthermore, CNN-FB learns its visual encoder from scratch using 224×224 image inputs, whereas SWM benefits from externally pretrained DINOv2 features. Although the reported computational efficiency comparison measures target-domain FLOPs, it excludes the external pretraining cost of DINOv2 and may be influenced by the relatively high input resolution and network configuration of CNN-FB.
Questions for authors

Q1. Can the authors provide more controlled ablations to distinguish the respective contributions of feature normalization, dimensionality reduction, and FB-objective-driven projector learning? In particular, how much of the improvement comes from learning the projector rather than applying simpler feature transformations?

Q2. Could the authors further clarify how Dreamer-v3 and TD-MPC2 were adapted to the reward-free pretraining and downstream zero-shot control setting? In particular, why were terminal value estimation and policy-prior trajectories disabled during CEM planning, and how might these choices affect baseline performance?

Q3. Can the authors provide additional CNN-FB results using its conventional lower image resolution, reporting both zero-shot control performance and computational cost? This would help determine whether SWM retains its computational efficiency advantage against a more computationally competitive CNN-FB configuration.

6 comments

We thank the reviewer for the constructive feedback. The additional experiments strengthen the evidence for the adaptor design and address the baseline and efficiency comparisons. Complete results are provided in C1–C4 of our common response.

1. Contribution and the role of projector learning — W1, Q1

Q1: The new C2 controls show that simpler feature transformations account for a substantial part of the improvement, while learning the projector provides additional, domain-dependent gains. At matched dimensionality and with LN held fixed, learning the projection using the FB objective improves returns over the frozen random projection by 69.0% on Cheetah, 15.5% on Walker, 15.4% on Quadruped, and 219.7% on Jaco (3.2× the return). Thus, learning provides particularly large gains on Cheetah and Jaco, while simple transformations already recover much of the improvement on Walker and Quadruped. The learned projection without LN also outperforms PCA in all four domains, and adding LN further improves its returns.

W1 novelty: SWM contributes a methodological extension of reward-free FB that replaces visual-encoder training from environment data with fixed pretrained visual knowledge. Our controls identify the sources and scope of its gains: the random-ViT control supports the value of visual pretraining, while competitive R3M results show that SWM is not tied to DINO (C1); naive integration fails, as raw DINO features yield near-zero returns, while fixed compression makes them usable and FB-driven projector training adds further gains(C2); and the same frozen features and adaptor improve FB in every domain but degrade TD-MPC2 on all locomotion domains, suggesting that the benefit is specific to successor-measure learning (C4). This design achieves higher mean returns than CNN84 in all four domains using approximately 43% of its target-domain FLOPs (C3). We will foreground this method extension and its new empirical findings, condense the FB preliminaries, and clearly attribute the inherited factorization and inference machinery.

2. Predictive-model comparisons and computational efficiency — W2, Q2 and Q3

Q2: To keep the train–test protocol aligned with zero-shot RL, we disabled terminal value estimation and policy priors: both terminal value and policy prior are learned from rewards, which are unavailable during pretraining, and learning them at test time would amount to downstream policy optimization, which the protocol excludes for all methods, including SWM. To compensate, we use a 15-step planning horizon, matching Dreamer-v3’s imagination horizon. At evaluation, their representations and dynamics remain frozen; the same budget of reward-labeled observations available to SWM are used to fit a scalar reward predictor. Actions are then selected through 15-step CEM planning.

With the same frozen DINO features and adaptor, SWM beats CNN-FB in all four domains (C3), whereas TD-MPC2 beats its CNN version only on Jaco and falls below it on all three locomotion domains (C4). DINO-based TD-MPC2 also stays below SWM in all four domains. This suggests that DINO and the projector do not bring a gain by themselves; the gain comes from combining them with successor-measure learning.

The DINO WM CLS baseline provides a complementary predictive-planning comparison using the same frozen visual features on Jaco (goal image needed), where SWM also outperforms predictive planning.

Q3: CNN84 offers performance comparable to CNN224 at substantially lower training cost (C3), making it the more competitive efficiency reference. Even against this stronger baseline, SWM still achieves higher mean returns in every domain. Its compute advantage is 2.34× in FLOPs and 3.5–3.8× in update wall-clock time, compared with about 5× against CNN224. We will update the paper’s performance and efficiency tables to include CNN84, use it as the primary reference for the efficiency comparison, and revise the “at least 5×” statement accordingly.

Regarding the pretraining cost of DINOv2: the pretrained encoder is treated as an off-the-shelf component rather than as part of the method’s training cost. DINOv2 is a public checkpoint, trained once and independently of this work. SWM keeps DINOv2 frozen and reuses the same checkpoint across all domains and tasks, so applying SWM to a new domain costs only the target-domain training measured in Table 4, whereas CNN-FB must train its encoder from scratch for every domain.

AuthorsOpenReview ↗

Follow Up to your Review

Dear Reviewer UGDn,

Thank you again for the time and effort you put into reviewing our work!

We understand that this might be a super busy period for you. We wanted to briefly let you know that we have posted point-by-point responses to your questions and concerns.

As the rebuttal period is ending very shortly, we would be grateful for your feedback on whether the responses address your concerns. We would be happy to clarify anything that remains open.

Thank you!

Best,

The Authors of Submission 47327

AuthorsOpenReview ↗

A Quick Summary of New Experiments for Q1–Q3 & Follow-Up

Dear Reviewer UGDn,

As the discussion period is closing soon, here is a short summary of how the new results (C1–C4) answer your questions:

  • Q1 – The FB objective matters, not just normalization: With LN and dimensionality held fixed, the FB-trained projector beats a frozen random projection in every domain (+69% Cheetah, 3.2x Jaco) and also beats PCA (C2).
  • Q2 – The gain comes from successor-measure learning, not from DINO alone: With the same frozen DINOv2 features and adaptor, SWM outperforms TD-MPC2 in all four domains, while DINO even hurts TD-MPC2 on locomotion (C4). Terminal values and policy priors were disabled because learning them requires rewards, which the zero-shot protocol excludes for all methods.
  • Q3 – The efficiency advantage holds at standard resolution: SWM beats CNN84 in every domain with 2.34x fewer FLOPs and 3.5–3.8x faster updates (C3).

We believe these results resolve the concerns raised in your review, and we would be grateful if you would consider updating your assessment.

Best regards, The Authors of Submission 47327

Reviewer_bXfzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors tackle the problem of zero-shot visual control. They propose SWM, which is a successor-measure world model that builds on a frozen DINO encoder, a lightweight trained adaptor and forward and backward successor-measure representations. At inference, new tasks can be specified by observation-reward pairs, and these can be directly used to extract a policy. The method is pretrained online without rewards and evaluated on pixel-based control tasks and the authors show that the proposed method, SWM, outperforms visual features trained from scratch with a CNN.

Critical strengths
  • The paper shows that frozen DINO with a linear adapter works for zero-shot RL with online FB, whereas naively using the raw DINO features fails. The performance of the method with DINO degrades less than CNN-FB's, when camera and color are perturbed.
Critical weaknesses
  • The contribution of the paper is limited. Essentially, the argument is that using a pre-trained DINO model as the feature extractor outperforms a CNN trained from scratch on zero-shot RL from pixels, when combined with successor measures. This is not a surprising result, as DINO has been extensively used for world modeling already. The paper shows that an adapter is needed, but the role of the training is not fully clear, as normalization alone is not ablated. The paper mostly focuses on excluding potential explanations.
  • The paper emphasizes that the DINO-based model is substantially more compute-efficient than the CNN. However, using a 224 x 224 image resolution is abnormal (e.g. DrQ uses 84x84) and is most likely the explanation. The TD-MPC2 and Dreamer baselines are structurally unsuitable for this setting, as they're missing a policy prior and a value function for the terminal value and these cannot be learned without rewards. More relevant baselines could be adaptations of TD-JEPA (https://arxiv.org/abs/2510.00739), HILP (https://arxiv.org/abs/2402.15567) (e.g. trained on SWM's own replay buffer), or using the DINO for world modeling (DINO-WM, https://arxiv.org/abs/2411.04983) to isolate what is the role of the FB objective and the frozen DINO features.
Questions for authors
  1. If you do SWM-raw with normalization only (potentially also with a random projection), how is the performance?
  2. Did you tune the CNN's image resolution? What is the difference in wall clock time?
  3. Did you try other frozen encoders than DINO?
  4. How does a DINO-WM-style predictive model on the same frozen features perform compared to the proposed SWM?
5 comments

We thank the reviewer for constructive feedback prompting closer examination of our design. New experiments address both central concerns below and provide new evidence for further discussion; full results are in C1–C4 of our common response.

1. Visual priors for zero-shot control—W1, Q1, Q3

W1: A pretrained prior’s benefit is not plug-and-play: DINO-based TD-MPC2 has lower mean returns than its CNN counterpart on all three locomotion domains (C4). Prior DINO-based world models such as DINO WM predict future features and plan toward a goal image; SWM learns a successor measure on frozen features without test-time planning.

Q3: With the same FB objective, random ViT features yield substantially lower mean returns than pretrained DINO, supporting visual pretraining. R3M is competitive with DINO on three of four domains; CLIP, though weaker than DINO, clearly outperforms random ViT in every domain, showing SWM is not tied to one pretrained encoder (C1). We chose DINO for its generality as a visual prior; R3M is pretrained on egocentric video for robot manipulation, and CLIP only with image–text alignment.

Q1: C2 separates normalization, random projection and FB-driven adaptation, showing complementary benefits. LayerNorm (LN) alone leaves Cheetah and Jaco at SWM-raw levels and yields under half of full SWM’s return on Walker and Quadruped. Frozen random projection with LN substantially outperforms LN alone. At matched dimensionality, FB-trained projection further improves returns in every domain, most on Cheetah (+69%) and Jaco (3.2× the frozen-projection return). The learned adaptor outperforms PCA in every domain; adding LN further improves the learned projection in all four.

2. Efficiency and predictive-model comparisons—W2, Q2, Q4

W2:

CNN resolution: see Q2.

TD-MPC2 and Dreamer-v3 baselines. We agree: lacking a task-specific policy prior and terminal value limits these baselines under our protocol. All methods receive the same reward-labeled observation budget, only at test time. Our reward-free adaptations fit a reward predictor over frozen dynamics using these labels, without task-specific actor–critic training. We include these state-of-the-art predictive world models(also used as baselines in DINO WM), to show what reward-free predictive pretraining achieves under this protocol. Both TD-MPC2 variants (CNN and DINO; Q4) share this limitation, so their comparison tests encoder choice in the same predictive-control setup.

HILP and TD-JEPA. New FB encoder comparisons (C1, C3), frozen versus FB-trained adaptors (C2), and predictive versus successor-measure control comparisons with matched pretrained inputs (C4) directly address visual priors and control learning. HILP and TD-JEPA are relevant offline zero-shot methods. Training them on SWM’s replay would provide a fixed-data comparison, but not evaluate the interaction of policy learning and online data collection. Matching both our online setting and visual inputs would require integrating these methods into an online exploration loop and adapting their visual front ends. We therefore prioritized these controls, which establish complementary benefits of pretrained visual priors and FB-driven adaptation in SWM, and show higher mean returns than the tested predictive alternatives with the same pretrained visual inputs.

Q2: CNN resolution and efficiency. CNN84 performs comparably to CNN224 at substantially lower training cost (C3), making it the more competitive baseline. SWM still achieves higher mean returns in every domain, with 2.34× fewer FLOPs and 3.50–3.83× faster parameter updates than CNN84 (C3).

Q4: Predictive modeling with the same visual representation.

DINO WM CLS: With the same frozen DINO CLS representation, SWM achieves 64.19 on Jaco versus 47.60 for DINO WM CLS (C4), supporting SWM’s design. We test this baseline only on Jaco, where its required goal images can be provided; the locomotion tasks lack meaningful goal images.

DINO-based TD-MPC2 provides a predictive baseline on the same frozen features in all four domains, with SWM’s frozen DINO encoder and adaptor architecture. SWM outperforms CNN-FB in all four domains (C3); DINO-based TD-MPC2 outperforms its CNN counterpart only on Jaco and underperforms on all three locomotion domains (C4). SWM also achieves higher mean returns than DINO-based TD-MPC2 in all four domains. This contrasting pattern shows that adding the same visual front end does not consistently improve control, suggesting that how these features are adapted and used for control matters. With C2’s adaptor ablations, these results support SWM’s design; the shared-feature predictive comparisons show its effectiveness as a complete control framework.

The revision will include: encoder and adaptor controls (C1, C2), CNN84 as the performance and primary efficiency reference in the tables (C3), and re-trained TD-MPC2 baselines with shared-feature predictive comparisons (C4).

AuthorsOpenReview ↗

Follow Up to your Review

Dear Reviewer bXfz,

Thank you again for the time and effort you put into reviewing our work!

We understand that this might be a super busy period for you. We wanted to briefly let you know that we have posted point-by-point responses to your questions and concerns.

As the rebuttal period is ending very shortly, we would be grateful for your feedback on whether the responses address your concerns. We would be happy to clarify anything that remains open.

Thank you!

Best,

The Authors of Submission 47327

AuthorsOpenReview ↗

A Quick Summary of New Experiments for Q1–Q4 & Follow-Up

Dear Reviewer bXfz,

Thank you again for your time and constructive review! As the discussion period is closing soon, we wanted to provide a quick summary of the new experiments we ran in response to your questions (Q1–Q4):

  • Q1 (Normalization & random projection): Ablated in C2. LayerNorm alone leaves Cheetah and Jaco at SWM-raw levels; LN + frozen random projection helps, and the FB-trained adaptor further improves returns in every domain (+69% on Cheetah, 3.2x on Jaco).
  • Q2 (CNN resolution & wall-clock): Evaluated in C3. CNN84 matches CNN224 at lower cost, and SWM still outperforms CNN84 in every domain with 2.34x fewer FLOPs and 3.5–3.8x faster updates.
  • Q3 (Alternative encoders): Evaluated in C1. R3M is competitive with DINO, and CLIP clearly outperforms a random ViT in every domain, showing SWM is not tied to a single pretrained encoder.
  • Q4 (Predictive models on shared features): Evaluated in C4. SWM outperforms DINO-WM on Jaco (64.19 vs. 47.60) and DINO-based TD-MPC2 in all four domains. Notably, DINO-based TD-MPC2 even falls below its CNN counterpart on locomotion, showing that DINO features are not plug-and-play.

If these results address your concerns, we would be grateful if you would consider updating your assessment. We are happy to clarify anything else before the deadline.

Best regards, The Authors of Submission 47327

Comments on the paper

3 comments
AuthorsOpenReview ↗

Common response Part 3

Planned revisions.

(1) Add CNN84 to the performance and efficiency tables and revise the efficiency claim to the numbers against CNN84.

(2) Replace the TD-MPC2 column of Table 1 with the corrected results, and add the DINO + adaptor variant and the DINO WM comparison on Jaco.

(3) Add the encoder comparison (C1) and the adaptor ablation (C2), and revise Section 5.3 to separate the effect of a fixed projection from FB-trained adaptation.

(4) Report standard deviations for all new results in all 16 individual tasks.

AuthorsOpenReview ↗

Common response Part 2

C3. SWM outperforms CNN84 with more efficient updates

Table 3: CNN baselines performance

MethodCheetahWalkerQuadrupedJaco
SWM234.77311.44558.1464.19
CNN84121.85242.73532.3148.70
CNN224158.90259.15476.0449.37

The two CNN resolutions have domain-dependent trade-offs: CNN224 performs better on Cheetah and Walker, CNN84 performs better on Quadruped, and Jaco returns are close. SWM achieves higher mean returns than both configurations in every domain.

Table 4. Computation and wall clock time.

MethodComputation (PFLOPs)Wall-clock time of updating parameters, min–max over domains
SWM52.532.76–3.02 h
CNN84122.9510.31–10.70 h
CNN224276.5422.51–23.45 h

All efficiency measurements use batch size 128, an NVIDIA RTX 2080 Ti, and matched interaction and update budgets. External DINO pretraining is outside this target-domain accounting. CNN84 substantially reduces computation while retaining competitive control performance, making it the more demanding efficiency reference. SWM still uses approximately 43% of the computation (2.34× fewer FLOPs) and spends 3.50–3.83× less wall-clock time on parameter updates. Together with the encoder controls, these results support concentrating SWM on adapting and exploiting frozen visual priors for control.

C4. Predictive baselines with the same visual representation

1. DINO Predictive baselines and its counterparts

Correction: the TD-MPC2 results in the submitted paper used goal-state information during pretraining. We re-trained TD-MPC2 under the same reward-free protocol as SWM; the numbers below replace the TD-MPC2 column of Table 1. To isolate the effect of the visual prior within TD-MPC2, we also compare its CNN encoder with frozen DINO features plus an adaptor (same structure as in SWM).

DomainTD-MPC2: CNNTD-MPC2: DINO + adaptorDreamer-v3: CNNSWM
Cheetah120.8825.50134.16234.77
Walker111.5089.5093.19311.44
Quadruped182.80156.90137.98558.14
Jaco15.6037.6033.5864.19

The TD-MPC2 comparison isolates the effect of the visual front end within predictive control: CNN achieves higher returns in all three locomotion domains, while DINO improves Jaco. Together with the consistent gains of DINO-based SWM over CNN-FB (C3), this contrasting pattern suggests that SWM makes more effective use of DINO representations, supporting the value of coupling pretrained perception with FB-driven adaptation and successor-measure control.

Without reward signals, TD-MPC2 and Dreamer learn weaker representations and have no policy prior or terminal value, which limits their performance. This follows from the reward-free protocol and applies equally to both TD-MPC2 variants, so the CNN-versus-DINO comparison within TD-MPC2 is unaffected. Unless noted otherwise, all new results are means over 3 seeds; standard deviations will be added in the revision. Our motivation for including these World Models as a baseline is that they represent state-of-the-art work, as they are also selected as baseline for DINO WM.

2. Goal based DINO predictive baselines

Our DINO WM CLS baseline uses the same frozen DINO CLS representation as SWM, using the CLS implementation from DINO WM. Jaco provides goal images in our evaluation protocol, obtaining 47.60 on Jaco versus 64.19 for SWM; locomotion tasks specify rewards without corresponding goal images. This result supports the practical value of learning long-horizon successor structure for direct policy extraction from frozen visual features.

AuthorsOpenReview ↗

Common response Part 1

We thank all reviewers for the constructive feedback, which prompted us to examine our design more closely. We summarize the additional experiments addressing common concerns below and discuss reviewer-specific questions in our individual responses. Unless noted otherwise, all new results are means over 3 seeds; standard deviations will be added in the revision.

C1. SWM benefits from DINO and can use other frozen visual priors

The new experiments clarify how SWM turns pretrained visual information into effective online control. DINO provides the strongest representation overall. R3M is on par with it on three of four domains, while CLIP, trained only with image–text alignment, is weaker on Quadruped and Jaco.

Table 1 Visual priors. The encoder comparisons give the following mean returns over 3 seeds for new experiments:

EncoderCheetahWalkerQuadrupedJaco
DINO234.77311.44558.1464.19
R3M113.59316.24510.5262.01
CLIP165.83267.81347.1319.33
Random ViT17.75160.2996.538.84

DINO achieves the highest mean return in three domains, while R3M slightly leads on Walker and is competitive on Jaco. All encoders share the same successor measure learning objective, isolating the effect of the visual representation; random ViT provides the no-pretraining control.

C2. SWM benefits from both fixed projection and learned adaptation

Table 2 Adaptor design. The experiments separate the contributions of feature transformation and normalization, 3 seeds for new ablations:

ConfigurationCheetahWalkerQuadrupedJaco
SWM-raw2.5753.0693.252.64
LayerNorm only2.41121.91244.332.53
Frozen random linear + LN138.90269.61483.5720.08
PCA140.52204.04484.0418.05
Learned linear without LN195.74260.54520.1342.49
Full SWM234.77311.44558.1464.19

Compression provides substantial gains even with frozen projection weights: frozen random projection with LN improves over LN alone in every domain and achieves returns close to PCA on three domains.

Learning the transformation provides further benefits. With dimensionality and LN retained, learning the projection improves over its frozen counterpart throughout. The projector receives gradients exclusively from the FB objective, isolating the additional benefit of FB-driven adaptation. Adding LN to the learned projection yields the highest returns in all four domains.

PCA chooses a fixed compression according to visual-feature variance. The learned adaptor's consistent advantage over PCA supports learning the compression for control. Together, these comparisons validate the complete adaptor design and isolate the contributions of its components.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.