acceptodds
Under review as a conference paper at ICLR 2027

CLAW: Continuous Latent Action World Models via Adversarial Latent Regularization

Abstract

We introduce CLAW, a fully end-to-end self-supervised framework for learning a world model jointly with continuous latent action representations directly from action-free videos. Our approach leverages adversarial latent regularization to prevent the latent from encoding future observations, so that it captures the structure of the action. The diffusion-based world model then models environment dynamics as a distribution over future observations rather than as a single deterministic rollout. Both components are learned from observation sequences alone without any action annotations. By simultaneously training the Latent Action Model and world model, CLAW learns to reason about how inferred actions induce environment transitions from visual observations alone. We show that the resulting latent action world model supports both imitation learning from observation and goal-directed planning. In imitation learning, latent actions extracted from raw videos enable behavior cloning. For planning, CLAW generates sequences of latent actions and maps them to executable actions to reach desired goals. Extensive experiments across diverse tasks and embodiments demonstrate that CLAW produces semantically meaningful latent action representations, supports effective action transfer, and enables planning and imitation from observation, outperforming existing methods.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Latent Action World Model, Latent Action Representation Learning, Generative World Model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_oVnvOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes CLAW, a self-supervised framework that jointly learns continuous latent actions and a diffusion-based world model from videos without action labels. A latent action model infers actions from consecutive observations, while the world model uses them to predict future frames. The main methodological component is a two-path adversarial regularizer: one prediction path receives the initial observation and latent action, while the other receives only the latent action and uses gradient reversal to discourage leakage of future-observation information. The paper evaluates the learned representations through imitation from observation, action retrieval and transfer, and real-robot policy pretraining, as well as visual planning with the world model. The authors report gains over several latent-action baselines on many tasks, although results vary across individual benchmarks.

Critical strengths
  1. The paper addresses two meaningful challenges in latent-action learning: future-observation leakage and uninformative representations. The problem is well motivated and clearly explained.
  2. The proposed method is innovative, particularly its joint training of the latent action model and world model with a two-path adversarial regularizer.
  3. The evaluation covers a wide range of tasks and includes useful visualizations and analyses. This breadth makes the empirical findings persuasive.
Critical weaknesses
  1. Action information can help predict future observations even without the current observation; absolute-position commands, for example, predict the robot’s next configuration. The regularizer may therefore suppress both scene information and useful action information. The transfer results and leakage metric do not sufficiently distinguish these effects.
  2. CLAW w/o Adv retains the second prediction branch’s updates to the shared world model. The comparison supports the benefit of adversarial gradients within this architecture, but does not establish an advantage over conventional joint training with only conditional prediction.
Questions for authors
  1. Can the authors provide evidence that the adversarial term reduces scene information while preserving useful action information? I suggest using independent probes to measure both types of information in the latent representation and examining how they change with adversarial strength.
  2. Does the full method consistently outperform conventional joint training with the second prediction branch entirely removed? Adding this baseline and extending the adversarial ablation to representative imitation-learning and real-robot tasks would clarify the contribution and its scope.

I will reassess and adjust my score based on the authors’ responses and additional experimental evidence.

3 comments

We thank the reviewer for the constructive review and the concrete suggestions.

Does the regularizer suppress useful action information, such as absolute-position content, along with scene information?

App. D.3.5 gives our conceptual argument: the auxiliary branch receives only ziz_i, so its loss can decrease only through information that predicts this particular future without the current observation. Absolute end-effector content generalizes across scenes, whereas object identity, layout, and appearance do not, so the gradient pushes primarily against scene-specific content.

Three pieces of evidence indicate that action content survives the regularizer:

  • Action transfer (Sec. 4.4.2, Fig. 4). Latents transferred to a different initial frame reproduce the source end-effector trajectory, matching what an absolute action space would produce, while the objects respond according to the new scene.
  • Retrieval (Sec. 4.3.4, Fig. 3). Nearest neighbors in CLAW's latent space share end-effector motion while object identity varies.
  • Controllability and planning (General Response). If the adversary stripped action content, the world model's dependence on zz and the planner's ability to ground latents to executable actions would both fall. Instead, CLAW has the highest Δt\Delta_tPSNR and planning success of every same-architecture variant, including the model trained without any adversary.

As D.3.5 acknowledges, these results do not isolate the absolute component directly, and we have not tested action spaces in which the agent controls nearly the entire scene. There, a uniformly applied penalty could suppress action content along with context. We will state this limitation more prominently.

Does the full method outperform conventional joint training with the second branch removed?

Yes. We trained CLAW w/o Branch, which is exactly this baseline (see the General Response).

Following your suggestion, we probed the latent with independent linear probes across adversarial strengths (see our follow-up to the General Response). As the penalty approaches full strength, scene content in zz declines while action accuracy reaches 0.71 by step 69k. This is consistent with the regularizer reducing scene information without suppressing action information.

Thank you for conducting the additional experiments and the linear probing analysis. These results directly address my main questions regarding the preservation of action information and the baseline without the auxiliary branch. I am satisfied with the response and will maintain my positive score

Reviewer_AYBVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CLAW proposes a self-supervised framework for jointly learning continuous latent actions and a diffusion world model from action-free videos. A ViT encoder extracts 32d latent actions from frame transitions, while a U-ViT predicts future observations conditioned on these latents. The main contribution is an adversarial regularization scheme that uses an auxiliary prediction branch with gradient reversal to discourage future-observation leakage. The learned model is evaluated on imitation learning, visual planning, and real-robot policy pretraining, reporting improvements over several existing latent-action methods.

Critical strengths
  • A simple approach to a relevant problem. Future-observation leakage is a fundamental challenge in latent action learning. The proposed adversarial regularization offers an alternative to conventional bottleneck-based approaches while allowing the latent action model and diffusion world model to be trained jointly.

  • Comprehensive downstream evaluation. The paper evaluates both latent action representations and world-model utility across imitation learning, visual planning, and real-robot manipulation. The results show consistent improvements on several benchmarks, while the additional controllability, leakage, and action-transfer analyses provide useful insights into the learned representations.

Critical weaknesses
  • The central adversarial mechanism is not convincingly validated. The adversary penalizes future-frame information decodable from the latent alone, which neither guarantees removal of context-dependent leakage nor preserves useful action information predictive of the future (App D.3.5). Moreover, removing the adversarial term yields weaker controllability but lower measured leakage in three of four environments (Table 5). The results do not appear to establish that the gains arise from leakage suppression.

  • Insufficiently controlled ablations and comparisons. The only direct ablation, “w/o Adv,” disables the adversarial gradient but retains the auxiliary branch and its updates to the shared world model. It is also absent from imitation learning (Table 1) and real-robot pretraining (Table 3). Comparisons with prior methods are largely apples-to-oranges and make it difficult to isolate the contribution of adversarial regularization.

  • Missing regularization baselines and limited evidence for continuous action learning. CLAW uses a fixed 32-dim normalized latent without capacity ablations. Controlled comparisons against alternative regularization strategies, including noise, sparsity, and KL regularization [1], or effect-alignment objectives [2], are missing.


[1] Garrido, Quentin, et al. "Learning latent action world models in the wild." Forty-third International Conference on Machine Learning. 2026.

[2] Jiang, Yuxin, et al. "Olaf-World: Orienting Latent Actions for Video World Modeling." Forty-third International Conference on Machine Learning. 2026.

Questions for authors
  1. Adversarial mechanism. Why should suppressing future-frame prediction from ziz_i alone prevent leakage that is recoverable jointly from (oi,zi)(o_i, z_i)? Can the authors directly probe the future-frame or scene information retained in the latent, with and without the adversary?

  2. Clean ablation. Can the authors remove the auxiliary branch entirely and evaluate this variant on imitation, planning and real-robot pretraining? Why does the current “w/o Adv” model show lower leakage but substantially weaker controllability?

  3. Regularization comparisons. How does the proposed objective compare with KL, noise or sparsity regularization [1], or effect-alignment regularization [2], under the same architecture and training budget? How sensitive are the results to latent dimensionality?

  4. Baseline fairness. Were the baseline models retrained under matched conditions? How are differences in pretraining and model capacity controlled, particularly for the real-robot experiments and the reused AdaWorld planning results in Table 4?

  5. Scalability and pretrained priors. CLAW trains a relatively small diffusion world model without pretrained generative priors. VideoWorld 2 [3] shows that a pretrained video diffusion decoder can relieve appearance–dynamics entanglement by taking over appearance modeling. Does the proposed adversarial regularization remain beneficial with a pretrained video diffusion backbone?


[1] Garrido, Quentin, et al. "Learning latent action world models in the wild." Forty-third International Conference on Machine Learning. 2026.

[2] Jiang, Yuxin, et al. "Olaf-World: Orienting Latent Actions for Video World Modeling." Forty-third International Conference on Machine Learning. 2026.

[3] Ren, Zhongwei, et al. "Videoworld 2: Learning transferable knowledge from real-world videos." arXiv preprint arXiv:2602.10102 (2026).

3 comments

We thank the reviewer for the thorough and thoughtful review. It prompted the experiments summarized in the General Response, which we believe substantially clarify the contribution.

Adversarial mechanism.

The auxiliary branch is a training signal that targets one specific shortcut: the latent encoding the future frame itself (the Future-copy anchor in Table 5). It does not detect information useful only jointly with oio_i, and we do not claim it does. That joint, context-dependent leakage is what our leakage metric evaluates (App. D.3.1), by holding zjz^j fixed, swapping the context through the world model, and measuring how much prediction quality depends on the source context.

The new experiments also reveal a role of the schedule that Sec. 3.2 understated. Unregularized joint training does not end up in the future-copy shortcut (its leakage stays far below the Future-copy anchor), consistent with Nikulin et al. (2025), who found no evidence of next-observation copying with high-dimensional observations; instead, the world model largely ignores the latent. As in Sec. 3.2, the two components address the two failure modes separately, and we interpret their roles as follows. The gradient reversal penalizes shortcut content, i.e., whatever in zz predicts oi+1o_{i+1} without oio_i. The schedule prevents under-use: the main loss can counteract the adversary only once the world model reads zz, so the schedule ramps the penalty up from near zero, letting that dependence form first. The LAM then sheds content redundant with oio_i while keeping transition-specific information. Together they raise Δt\Delta_tPSNR by 1.5–5.1 dB over the no-branch baseline, with leakage comparable or lower on three of four environments (see also our response to Reviewer F9nV). We will revise Sec. 3.2 accordingly.

Clean ablation.

We trained CLAW w/o Branch with the auxiliary branch removed entirely (see the General Response). It behaves like w/o Adv, with weak controllability and much lower planning success than CLAW, so the retained branch is not what limits w/o Adv. As for the lower leakage of w/o Adv: leakage is uninformative when the world model barely uses the latent, since matched- and cross-context errors are then similar by construction (Sec. 4.5; App. D.3.1; Fig. 18). Leakage must therefore be read jointly with Δt\Delta_tPSNR.

Regularization comparisons.

We compare against VQ (LAPO's codebook settings) and KL (β\beta-VAE, AdaWorld's β\beta) bottlenecks that use CLAW's LAM and world model unchanged and are trained end to end on the same data and budget, so only the regularizer differs. Both drive the latent toward disuse (near-zero Δt\Delta_tPSNR) and plan substantially worse than CLAW; for VQ, this takes the form of latent collapse (see the General Response).

Baseline fairness.

  • Simulated benchmarks: all latent action world models are trained on the same data per environment (Sec. 4, Table 8), including the VP2 benchmark datasets (App. C.1). The exception is AdaWorld in Table 4: its VP2 results use a low-resolution (64×64) variant whose architecture and weights are neither specified in the paper nor publicly released, so we report the numbers from the original paper (marked *).
  • Real robot (Table 3): all policies, including Scratch, use the same ACT backbone (Zhao et al., 2023). For the latent-pretrained methods, only the prediction head changes to match each method's latent action size; the protocol is otherwise identical (Sec. 4.3.2). The methods differ only in the latent action model that supplies the pretraining targets.
  • Same-architecture comparison: the new rows use CLAW's LAM and world model unchanged, so parameter counts are identical up to the codebook or mean/log-variance head, and CLAW's adversarial pathway shares weights with the main world model (Fig. 1), adding no parameters. CLAW outperforms all of these rows in planning on every environment (General Response).

Scalability and pretrained priors.

We trained CLAW with a larger DiT-based world model in place of the U-ViT. Due to compute limits, we evaluated it qualitatively: its latent actions show retrieval and action transfer behavior similar to the U-ViT model (nearest neighbors share the same motion across visually different scenes, and transferred latents reproduce the source behavior from a new initial frame), and we will include these results in the supplementary material. We also expect the method to scale, since the adversarial term acts on the LAM through the gradient into zz and requires only that the world model condition on (oi,z)(o_i, z) in one pathway and on zz alone in the other, as the DiT variant illustrates. We have not tested a pretrained video diffusion backbone, which is beyond the rebuttal period; the same argument applies, since the regularizer does not depend on how the decoder is initialized.

Thank you for the additional experiments. I appreciate the authors' efforts to address the concerns raised in my review.

However, my main concern regarding the adversarial mechanism remains unresolved. Contrasting state-conditioned and state-free future prediction is a reasonable design, but it does not necessarily distinguish action information from appearance leakage. It remains unclear whether the proposed regularization actually reduces leakage or mainly helps the world model make better use of the latent actions.

The new ablations show encouraging improvements in controllability, but do not directly demonstrate leakage suppression. In particular, measured leakage is higher than the w/o Branch baseline in three of four environments, although I understand that this metric is difficult to interpret when latent utilization is low.

Overall, I find the approach interesting, but I am still not fully convinced by the evidence for the proposed mechanism. I therefore maintain my rating of 2.

Thank you for the follow-up and for taking the time to look at the new results.

We think the core of your concern is fair: the leakage metric goes through the world model, so on its own it can't tell us whether the regularizer removes appearance from the latent or just gets the world model to use the latent more. That is exactly what the linear probes in our follow-up to the General Response were meant to separate. They are trained on the frozen latent, with no world model involved, so they measure what zz actually contains, however much the world model relies on it.

What they show is that as the adversarial weight grows, less of the scene can be decoded from zz. Scene R2R^2 falls from about 0.4 to 0.27 and drops at each of the last five checkpoints, while action accuracy climbs to 0.71. Since the world model plays no part in this measurement, better utilization can't be what's driving the drop.

You're also right about the leakage numbers. CLAW's mean leakage is higher than w/o Branch on three of four environments. On Crafter and Climber the gap is within one standard deviation (0.28 ± 0.12 vs. 0.23 ± 0.17, and 0.51 ± 0.11 vs. 0.44 ± 0.11), which is why we called them comparable, but on Jumper it is clearly higher. As you point out, though, w/o Branch uses the latent far less, and leakage can only show up in predictions that depend on zz. The extreme case is a plain video model that never sees the transition: its leakage is zero by definition, yet it gives no control at all. Our VQ and KL variants come close to this in practice, with near-zero leakage and near-zero controllability. That is why the two metrics have to be read together. A fairer comparison is against methods that use the latent about as much. LAPO and CoMo reach similar or lower controllability than CLAW in Table 5, yet CLAW leaks less than both on every environment (for example, 0.28 vs. 0.42 and 0.44 on Crafter, and 0.17 vs. 0.35 and 0.86 on Ninja).

So we don't see these as competing explanations. The schedule gives the world model time to start using the latent, and the gradient reversal limits how much appearance comes along with it. We'll make that distinction clearer in the revision and include the probes over the full run.

Reviewer_F9nVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper learns a world model from videos with no action labels. Two parts are trained together: a latent action model, which looks at two frames and outputs a small vector (a “latent action”) describing what changed and a diffusion world model, which predicts the next frames from the current frame and the latent actions. To stop the latent from simply copying the next frame, the authors add an adversarial branch that penalizes predicting the next frame from the latent alone. The aim is for the latent to be useful only together with the current frame, so it describes the action rather than the future image. To act in an environment, predicted latents are converted to real actions by nearest-neighbor lookup in a small set of labeled transitions.

The method is evaluated in four ways. First, imitation from action-free expert videos: a policy predicts latent actions. Second, planning: the planner searches over latent actions with the world model. Third, policy pretraining on a real UR5 robot, using a latent action model trained on Something-Something v2. Fourth, analyses of the latent space. CLAW gives the best results on most benchmarks.

Critical strengths
  • The idea is simple and easy to add to other methods.
  • The evaluation is broad: many environments and a real robot.
  • Methods show strong empirical results in various setting (policy learning, planning, ..)
Critical weaknesses
  • The ablation doesn’t fully support the claim in Table 5. Section 3.2 says the adversarial term prevents leakage. Without it (Table 5), deltat-PSNR drops on all four environments (Crafter: 2.47 vs. 8.31) and leakage is lower on 3 of 4. This points to a latent that the world model ignores the latent, when the adversarial loss is not present ?

  • Paper lack of control comparison for the different latent information bottleneck method (VQ, Variational, Adversarial), where the rest of the systems (model, training data) is fix.

Questions for authors
  • How does CLAW compare with a KL or VQ bottleneck in the same model?
  • In Table 5, without the adversarial term, leakage is lower on 3 of 4 environments and detlat-PSNR drops (Crafter: 2.47 vs. 8.31). This suggests the latent is used less by the world model without an adversarial loss, rather than leaking more. How does the adversarial term make the world model rely more on the latent?
2 comments

We thank the reviewer for the positive assessment and the insightful questions.

The w/o Adv ablation suggests the world model ignores the latent without the adversarial loss. How does the adversarial term make the world model rely more on the latent?

We agree with your reading of Table 5, which is also our interpretation in Sec. 4.5: the low leakage of w/o Adv reflects weak dependence on the latent, not a cleaner representation. This observation also led us to clarify how Sec. 3.2 describes the two components. As designed, the gradient reversal removes the future-frame shortcut from the latent, and the schedule determines when that removal happens. Sec. 3.2, however, presents the schedule only as a safeguard that avoids inducing under-use. The new results show that under-use is the default outcome of joint training without the adversary, so the schedule's timing plays an active role in preventing it: by letting the world model come to depend on zz before the penalty takes effect, it keeps the world model reading the latent once the shortcut is gone. A new baseline with the auxiliary branch removed entirely confirms that under-use is the default. It behaves like w/o Adv (3.30 vs. 2.47 dB on Crafter) and plans far worse than CLAW (33.33 vs. 68.33%).

As for why, we believe the answer lies in the interaction between the adversary and its schedule (Eq. 2), which acts as a curriculum. The main prediction loss can counteract the adversary only if the world model already reads zz, since its gradient with respect to zz is large only when predictions depend on the latent. Early in training δ≈0\delta \approx 0, so the latent is unconstrained, possibly including future-frame content, and the world model learns to read it. As δ\delta rises, the LAM receives two opposing signals. The reversed gradient penalizes content in zz that helps predict oi+1o_{i+1} without oio_i; since consecutive frames are largely identical, this is mostly content that duplicates oio_i, which the main pathway does not need. The main loss, which now depends on zz, penalizes removing what the world model cannot obtain elsewhere: the transition itself. The LAM resolves the two by shedding redundant content while keeping transition-specific information.

Without the adversary, nothing separates the two kinds of content. Because the world model can already predict most of oi+1o_{i+1} from oio_i, and the transition information in the latent is mixed with content that duplicates oio_i, the world model gains little from reading the latent and relies mostly on oio_i. Applied at full strength from the start, we expect the adversary to fail differently, removing the latent's content before the world model learns to read it, since the main loss would then offer little counter-pressure. This parallels KL annealing for avoiding posterior collapse in VAEs (Bowman et al., 2016) and the warm-up phase Wang et al. (2025) use in joint LAM–world model training. We will revise Sec. 3.2 accordingly.

How does CLAW compare with a KL or VQ bottleneck in the same model?

We trained both end to end with CLAW's LAM and world model unchanged, on the same data and budget, so the regularizer is the only difference: VQ uses LAPO's codebook settings and the KL bottleneck uses AdaWorld's β\beta. Both drive the world model to ignore the latent (near-zero Δt\Delta_tPSNR on every environment) and plan substantially worse than CLAW: VQ reaches 10–25% and the KL bottleneck 18–35% across environments, versus 57–68% for CLAW, and both stay near or below the random-action baseline on Procgen (for VQ, because of latent collapse; see the General Response).

The same regularizers fare better in their original setups: AdaWorld reaches 0.15–2.56 dB and LAPO 4.90–9.14 dB (Table 5). The regularizer settings are identical, so what differs is joint training with a stochastic diffusion world model, consistent with the decoder-strength explanation in the General Response. CLAW also outperforms both original methods in planning on three of four environments (Table 2).

Linear probes of the latent during training are consistent with the shedding step of the curriculum account in our response (see our follow-up to the General Response). Scene content in zz rises while the adversarial weight is still ramping up (from 0.27 to about 0.4 R2R^2) and falls back to 0.27 as the weight approaches full strength, consistent with an unconstrained latent early in training that later sheds content redundant with oio_i. Separately, action accuracy reaches 0.71 by step 69k.

Comments on the paper

2 comments

We thank the reviewers for their careful reading. As requested, we ran two new experiments: CLAW w/o Branch, standard joint training with only the conditional pathway (AYBV, oVnv), and same-architecture bottlenecks, which replace the adversarial branch with VQ using LAPO's codebook settings or a KL bottleneck using AdaWorld's β\beta (F9nV, AYBV). All variants use CLAW's LAM and world model and are trained end to end on the same data and budget. We also add a random-action planning baseline.

Controllability and leakage, same architecture (Δt\Delta_tPSNR ↑\uparrow / leakage):

VariantCrafterJumperClimberNinja
VQ0.089 / 0.0100.004 / 0.0020.139 / 0.0030.110 / 0.000
β\beta-VAE (KL)0.030 / 0.0060.043 / 0.0010.010 / 0.0100.013 / 0.006
CLAW w/o Branch3.304 / 0.2303.974 / 0.2964.371 / 0.4404.857 / 0.485
CLAW w/o Adv2.47 / 0.054.18 / 0.314.30 / 0.424.34 / 0.24
CLAW8.31 / 0.289.07 / 0.485.83 / 0.519.02 / 0.17

Visual planning success (%), same architecture:

VariantCrafterJumperClimberNinja
Random21.6731.6728.3326.67
VQ25.0013.3310.0018.33
β\beta-VAE (KL)28.3326.6735.0018.33
CLAW w/o Branch33.3328.3331.6735.00
CLAW w/o Adv26.6735.0038.3325.00
CLAW68.3363.3360.0056.67

What these results show.

The auxiliary branch alone does not explain the gains. CLAW w/o Branch and w/o Adv have similar controllability, and both plan far worse than CLAW. Adding the adversarial branch with its schedule raises Δt\Delta_tPSNR by 1.5–5.1 dB over w/o Branch and planning success by 22–35 points on every environment. Without the adversary, planning stays close to the random-action baseline on Procgen.

In our training, under-use of the latent is the first obstacle; once the latent is used, leakage becomes equally important. Prior work identifies two failure modes for latent actions: encoding exogenous content of the future observation (Nikulin et al., 2025; Lee et al., 2026), and collapsing to a representation the world model disregards (Wang et al., 2025). Without the adversary, our setting falls into the second. We attribute this to the strength of the decoder: the world model receives oio_i, which already explains most of oi+1o_{i+1}, so it can predict well without reading zz. Similar arguments appear for joint LAM–world model training (Wang et al., 2025) and for VAEs (Bowman et al., 2016).

Once the world model does use the latent, leakage becomes just as important, since any scene-specific content in zz now affects predictions. CLAW therefore needs the adversary, not only a mechanism that encourages use.

In our joint training, information bottlenecks, which target leakage, make under-use worse: with VQ or KL in the same architecture, the world model effectively ignores the latent (near-zero Δt\Delta_tPSNR), and planning stays near or below the random-action baseline on Procgen. Their near-zero leakage reflects this rather than a cleaner representation (App. D.3.1).

VQ collapses, which explains its below-random planning. Codebook perplexity, the effective number of codes in use, stays at 1.75–3.4. Planning goals come from uniformly random actions (App. C.4), which a collapsed latent cannot distinguish, whereas a random policy can execute every action.

The adversary increases controllability without a consistent leakage cost. Relative to w/o Branch, CLAW's leakage is comparable on Crafter and Climber, higher on Jumper (where Δt\Delta_tPSNR more than doubles), and substantially lower on Ninja.

Mechanism. The gradient reversal penalizes shortcut content in zz, and the schedule (Eq. 2) starts that penalty near zero and ramps it up, so the world model learns to read the latent first. The LAM then sheds content redundant with oio_i rather than abandoning the latent. The full account is in our response to Reviewer F9nV.

Revisions to the paper.

  • Sec. 3.2 will state that under-use is the default outcome without the adversary: the gradient reversal penalizes the shortcut, and the schedule, by ramping that penalty up from near zero so the world model reads zz first, prevents under-use.
  • Tables 2 and 5 will add the same-architecture variants; Table 2 will add the random-action baseline, episode counts, and standard errors.
AuthorsOpenReview ↗

Follow-up to the General Response: linear probes of the latent

As suggested by Reviewers oVnv and AYBV, we trained independent linear probes on the frozen latents across adversarial strengths. To save storage, we had not kept intermediate checkpoints of the original runs, so we reran CLAW training on Crafter, saving checkpoints throughout, and probed each one. The action probe predicts the ground-truth action (majority-class baseline 0.38; uniform guessing over Crafter's 17 actions gives 0.059); the scene probe reconstructs oio_i, downsampled to 8×8×38\times8\times3 (192 values), from zz, and R2R^2 is the fraction of its variance explained on held-out episodes (R2=0R^2 = 0 corresponds to always predicting the mean appearance). This follows established representation analysis methods (Alain & Bengio, 2016; Dosovitskiy & Brox, 2016; Gurnee & Tegmark, 2024).

Stepδ\deltaAction acc. ↑\uparrowScene R2R^2 ↓\downarrow
3k0.14890.3820.265
6k0.29130.4310.412
9k0.42190.5110.387
12k0.53700.6700.351
15k0.63510.6720.424
21k0.78180.6890.357
24k0.83370.6950.366
30k0.90510.6800.318
36k0.94680.6860.298
39k0.96030.6580.384
45k0.97800.6920.322
48k0.98370.6980.366
54k0.99100.7010.323
60k0.99510.7070.304
63k0.99630.7050.283
69k0.99800.7090.273

Scene content follows the two phases of the schedule. Scene R2R^2 rises from 0.27 at the first checkpoint to an average of 0.39 for 0.29≤δ≤0.640.29 \leq \delta \leq 0.64, then declines to 0.32 for δ≥0.9\delta \geq 0.9, with some fluctuation, and decreases at each of the last five checkpoints (0.366 to 0.273). This is consistent with what the schedule is meant to produce: the latent appears to take on scene content while the penalty is still ramping up and to shed it as the penalty approaches full strength.

Action content is preserved throughout. Action accuracy rises from the majority baseline to 0.67 by δ≈0.54\delta \approx 0.54 and reaches 0.71 by step 69k. Removing scene content therefore does not come at the cost of action information.

Training has not yet reached its full budget (69k of 100k steps). Because the discussion period is ending, we share these results now and will post the remaining rows as soon as training completes. The revised paper will include the complete, updated plot of both probes over the full run.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.