acceptodds
Under review as a conference paper at ICLR 2027

La-JEPA: Compact Robot Policies with Latent World Modeling

12 views

Abstract

World-action models commonly adapt pretrained video generators, and vision-language-action models build on pretrained vision-language backbones; both tie policy architecture and inference cost to large transformers designed for generation rather than control. While the knowledge accumulated through large-scale pretraining is certainly useful, it remains unclear how much generation-oriented architectural choices affect control performance, and whether the trade-off required to access this pretraining is worth it. In this work, we introduce **Latent Action JEPA (La-JEPA)**, a compact world-action model whose policy transformer is trained from scratch in the feature space of a pretrained V-JEPA 2.1 encoder, so that its architecture and capacity can be chosen for control. Following Fast-WAM, La-JEPA learns to predict future features jointly with actions during training and generates only actions at inference. **Among policies without embodied pretraining, La-JEPA achieves state-of-the-art clean and randomized success on RoboTwin 2.0 and is equivalent to the state of the art on LIBERO-Plus, while using far fewer parameters than the strongest competing policies.** It runs substantially faster than Fast-WAM at comparable performance and outperforms the closest JEPA-based methods, LeapBot-WA and JEPA-WAM, on LIBERO-Plus and RoboTwin 2.0 without their pretrained policy backbones. On four real-robot tasks, including contact-heavy pushing and long-horizon multistep manipulation, its average success is comparable to , outperforming non-embodied-pretrained baselines. Through a design study of the choices behind La-JEPA, we show that JEPA features outperform video VAE latents, most clearly out of distribution, and that training-time future prediction choices significantly impact the policy even when future prediction is skipped at inference. These results establish a compact, efficient foundation for JEPA-based world-action modeling: without a generative pretrained policy backbone, La-JEPA is comparable to or better than far larger policies built on video and language generation models, suggesting that robot foundation models may benefit from architectures designed for control rather than inherited from generation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
JEPA, Robotics, World Action Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jXRXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Latent Action JEPA (La-JEPA), a world-action model that has both world model DiT (future state predictor) and action DiT (robot action generator) jointly trained from scratch in a pretrained V-JEPA2.1 latent space. The model jointly predicts future states and actions using flow matching during the training. The action tokens are designed to attend to the current observation tokens, so that during inference the future state tokens can be dropped. The proposed method performs well over multiple simulation and real-world benchmarks like LIBERO-Plus and RoboTwin.

Critical strengths

The paper is clearly presented and easy to follow, the selected experiments are reasonable and the authors provided helpful details. Table 4 is informative with important ablations on action future prediction, latent representations and so on.

Critical weaknesses

The contributions of this paper need further clarification. Given the current state, it could be easily regarded as a natural extension of Fast-WAM and Faster-WAM as mentioned in the paper, or VLA-JEPA[1]. The claim of a compact, effective world-action policy, together with evidence to guide its design mainly comes from the large scale pretraining of V-JEPA and the jointly training of future prediction and action generation, neither of them is from this paper. I would appreciate further clarifications on this if I get it wrong.

[1] Sun, Jingwen, et al. "Vla-jepa: Enhancing vision-language-action model with latent world model." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026.

Questions for authors

No

3 comments

We thank the reviewer for their feedback. We also invite them to take a look at our general response, which reports updated scores on our benchmarks and adds full-RoboTwin results to show scalability.

On the contribution. We agree that the main novelty of La-JEPA is not architectural. The current trend in world-action models is to scale video generation pretraining. This is natural: it is something WAMs can exploit better than VLAs, as world-modeling objectives have been shown to make policies scale better with data (DriveVLA-W0 [Li et al., 2025]). Fast-WAM and Faster-WAM follow this direction and keep a pretrained video generator as the policy backbone. Our work shows that more of the value currently lies in an adapted latent space and in the regularizing effect of the world loss itself than in knowledge transferred from video generation pretraining: La-JEPA has no generative backbone and trains a compact policy from scratch, yet matches or exceeds these much larger models with faster inference speeds. No other policy without a pretrained VLM or video backbone comes anywhere near competitive performance, let alone the state of the art among policies without embodied pretraining, which La-JEPA reaches on both LIBERO-Plus and RoboTwin.

We will rewrite the introduction to state our contribution more clearly: not a new architecture, but evidence, against the field's trend, that a compact policy trained from scratch in an adapted latent space matches or exceeds policies built on large generative backbones, with the unified ablation identifying which ingredients matter.

On V-JEPA pretraining and joint training. We use both as ingredients and do not claim them as our contributions. The question we answer is which of them matters, and how. The design study isolates the latent space and the future-prediction objective, and the unified ablations added in this rebuttal confirm it under a single recipe (see our answer to Reviewer QK7p): replacing V-JEPA features with VAE latents costs 13.7 points on clean RoboTwin and drops randomized success below 5%, and removing future prediction costs 2 to 5 points on every benchmark, even though futures are never predicted at inference. We also note that V-JEPA 2.1 is pretrained on about 163M images and videos (VisionMix163M: 142M images and about 21M videos), substantially less than the billions of images and videos behind the video generators that WAMs such as Fast-WAM build on: La-JEPA's results therefore cannot be explained by more pretraining data; they point to the representation itself.

On VLA-JEPA. VLA-JEPA uses JEPA-style latent prediction to pretrain a VLA backbone on video, then fine-tunes an action head. La-JEPA does the opposite: it has no pretrained policy backbone and trains the policy directly in the adapted V-JEPA latent space, with future prediction as a joint training objective rather than a pretraining stage. La-JEPA S also outperforms it on LIBERO (97.9 vs. 97.2) and LIBERO-Plus (80.0 vs. 78.0, official text, episode-weighted; VLA-JEPA reports 79.5 as a category average), our B and L models extending that advantage. We will add this comparison to the related work.

I appreciate the authors' rebuttal, especially the new clarification on the contribution of this paper. I understand the paper suggests we should adapt latent space rather than work with large generative backbones, and V-JEPA is a good example. The experiment of V-JEPA features → Wan2.2 VAE latents supports that the latent space of V-JEPA is better than Wan2.2 VAE.

However, the current formulation of the paper looks more like a tech report, instead of a research paper. If the claim is to show the advantages of the latent space over generative backbones, I would expect to see a deeper analysis on the property of the latent space, e.g. what makes such a good latent space, what makes it different than a generative latent and so on. Though there are strong results shown by the current paper, answering these questions should make the paper more insightful to the community and inspire further research.

Considering all these, I would like to maintain my current score.

We thank the reviewer for their reply and for acknowledging the strength of our results. We realize our claim was not stated clearly enough and will clarify it in the final version: we do not oppose adapted latent spaces to pretrained backbones. Training from scratch is not integral to our method, but it is central to our claim: tailoring the policy's design, including its latent space, matters more than the knowledge transferred from a heavily pretrained generative backbone.

Which latent spaces suit world modeling is studied by dedicated work: V-JEPA 2.1 reports the state of the art among frozen encoders on prediction and world-model planning (Mur-Labadia et al., 2026), and a recent comparison of six encoders under a fixed world-model protocol ranks semantic encoders ahead of reconstruction ones, with "V-JEPA 2.1 (strongest overall on policy)" [1]. Our contribution is to show how far a policy can go by picking the best currently available space and designing the policy around it from scratch. Our feature-space comparison confirms this choice: V-JEPA features outperform Wan2.2 VAE latents in all four evaluations (Section 4.1, Figure 3B), with the largest LIBERO-Plus gap under background textures (63.0% vs. 36.0%, reply to Reviewer QK7p) and visual perturbations.

More generally, we understand the concern that the paper reads as a technical report: the text mostly puts forward the strength of our results, which obscures the novelty. It should emphasize two points:

First, most recent strong WAM contributions focus on scaling the video backbone and its data, but ignore the choice of latent space, as they are constrained by existing backbones made for video. We take the opposing view: the choice of latent space is just as critical, and even small video backbones trained from scratch can be competitive. This does not exclude pretraining as future work (limitations, Section 6, "Large-scale pretraining", p. 9).

Second, training the video backbone natively in that space is important: LeapBot-WA aligns the Fast-WAM architecture with V-JEPA 2.1, our space, and we still significantly outperform it. Previous non-VAE policies under-perform VAE-based baselines, sending mixed signals about the viability of moving away from VAEs. By reaching the state of the art with faster inference and fewer parameters, our work shows that designing or pretraining in spaces adapted to robotics is a more effective path than simply scaling data and model size.

In the final version, we will (1) rewrite the abstract and introduction around these two points, with pretraining in an adapted space as complementary to our approach; (2) open our analysis with the feature-space comparison (Section 4.1, Figure 3B), currently one ablation among many, and expand the discussion of what makes this space suited to control; and (3) discuss the pretraining scale behind each baseline's backbone. We hope this addresses the reviewer's remaining concerns.

[1] Nilaksh, et al. "Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models." Advances in Neural Information Processing Systems, Evaluations and Datasets Track, 2026.

Reviewer_QK7pOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

La-JEPA addresses the problem that current embodied world-action models (WAMs) mostly inherit the full video-generation backbone, leading to architectural redundancy and heavy training/inference costs. The paper proposes to use only the encoder module of a pretrained video model, paired with World and Action DiTs trained from scratch. The model jointly predicts future features and actions during training, and generates only actions at inference. It also proposes several techniques for optimization, including data augmentation, camera dropout, proprioception dropout, and encoder LoRA. The method runs substantially faster at inference and achieves strong results.

Critical strengths

S1. This is an important problem in embodied world modeling, and it challenges the current mainstream view. The paper uses only the encoder of a pretrained video model together with from-scratch-initialized DiTs, providing a new direction.

S2. The method achieves very good results in embodied control, especially in OOD generalization, reaching state-of-the-art or highly competitive performance across multiple benchmarks and evaluation settings: all three La-JEPA sizes outperform every baseline without embodied pretraining on RoboTwin 2.0; on LIBERO-Plus, La-JEPA L matches Cosmos Policy and ImageWAM with 4–5 times more deployed parameters; and on real-robot tasks its average success is comparable to the embodied-pretrained π0.5\pi0.5.

Critical weaknesses

W1. The paper runs many ablations, but the variable control is imprecise and the organization is somewhat confusing. La-JEPA performs best in OOD generalization, yet it actually stacks many tricks, each of which could significantly improve OOD performance, including image augmentation, camera dropout, and proprioception dropout. The ablation blocks in Figure 3 each use a different reference recipe, and the training data and evaluation protocols are not fully consistent across blocks, making it difficult to judge which modules truly matter under a unified configuration.

W2. The comparison with the main baseline JEPA-WAM is not fully controlled. Section 5.2 notes that JEPA-WAM is trained and evaluated on only 20 tasks, whereas La-JEPA is trained on all 50 tasks. The difference in training-task coverage could itself significantly affect the final results, especially OOD generalization, which weakens the persuasiveness of La-JEPA's reported advantage.

My main concerns are about evaluation rigor rather than the core idea, and I consider them fixable. If the authors can provide (1) a unified ablation isolating the contribution of the architecture from the augmentation/camera-dropout tricks, and (2) a JEPA-WAM comparison under matched training-task coverage, I would be willing to raise my score.

Questions for authors

Q1. Could the authors provide a more unified ablation under consistent training and evaluation settings? In particular, of the final OOD performance gains, how much comes from the proposed architecture and representation space, and how much from robustness tricks such as image augmentation and camera dropout?

Q2. Could the authors compare with JEPA-WAM under matched training-task coverage, or further explain how much the additional training tasks contribute to the reported performance gap?

Q3. Have the authors investigated why La-JEPA L underperforms La-JEPA B on RoboTwin 2.0? Specifically, would reducing the relative capacity of the Action DiT improve performance at the larger model size?

5 comments

We thank the reviewer for the precise requests. We also invite them to take a look at our general response, which reports updated scores on our benchmarks and adds RoboTwin full results to show scalability.

W1/Q1: We understand the reviewer's concern. The additive ablation was, in hindsight, a valuable way to choose components given our results, but it doesn't tell how much each part brings to the final recipe including interactions between components, which we agree is valuable information. We provide the requested ablation: we remove each component from the final S recipe one at a time, with everything else fixed (data, schedule, seeds, evaluation). Values are deltas in points against this reference, non significant numbers are marked "ns".

Removed from final recipeRoboTwin cleanRoboTwin rand.LIBEROLIBERO-Plus (official text)
V-JEPA features → Wan2.2 VAE latents−13.7−26.5−1.4−10.7
Future-prediction objective−4.8−2.9−2.2−4.5
Image augmentation−2.4−15.1−2.4−13.0
Camera dropout−0.7 (ns)+2.8−0.2 (ns)−4.0
Proprioception dropout−0.8 (ns)−1.2 (ns)+0.5 (ns)−1.0
Encoder LoRA−4.0−1.0−0.9−4.7
All three robustness components (aug., camera and proprio. dropout)−3.5−14.6−1.3−12.2
VAE latents and all three robustness components−25.1−30.5−9.4−43.4

Reference S: RoboTwin 85.6 / 33.1, LIBERO 97.9, LIBERO-Plus 79.5 (seed 42).

The representation and future prediction matter in every setting, while image augmentation brings most of the OOD robustness. These components cannot compensate for the representation: with them, a VAE-latent policy only matches a V-JEPA policy without them on LIBERO-Plus (68.9 vs. 67.3). Image augmentation is also standard among competitive baselines: π0.5, Cosmos Policy, ImageWAM and Qwen-RobotManip all train with crops, rotations and/or color jitter. Camera-pose conditioning (Section 3.1) brought no gain over three seeds, so the final recipe omits it. The requested ablation also shows that on RoboTwin, camera dropout interacts poorly with the rest of the recipe; we will reflect this in the text and figures. While the additive ablation was a good way to build the method, the ablation the reviewer suggested gives clearer insight into what each component contributes, so we will replace Figure 3 with it in the revision.

W2/Q2: We tried to be as generous as possible with JEPA-WAM to make sure we didn't bias the analysis in our favor, as we can't retrain their model on all 50 tasks since they don't release a RoboTwin training path. However, the reviewer's comment highlighted a very simple run that we missed and should have added: we train La-JEPA B on their 20 tasks only, which provides a clean comparison under their chosen setting:

ModelTraining tasksCleanRandomized
JEPA-WAM (reported)2079.936.9
La-JEPA B2095.256.5
La-JEPA B (3 seeds)5095.949.9

La-JEPA leads by 15.3 points in clean scenes and 19.6 in randomized ones. The additional tasks don't explain the gap: on these 20 tasks, training on all 50 actually lowers randomized success.

Q3: We believe the limiting factor is the training data rather than the size of the action expert. Two results support this: with three seeds, S, B and L are within seed spread on clean-only RoboTwin (about 50 demonstrations per task), and on the full RoboTwin set (about 10× more data), success increases monotonically with size (general response §3). We also ran the experiment the reviewer suggests: halving the Action DiT width at L does not help: it reaches 86.6 clean and 30.9 randomized, against 86.4 and 33.4 for the reference L. We will update the hypothesis in Section 5.1 accordingly.

Following the reviewer's ablation, which showed that camera dropout interacts poorly with the rest of the recipe on RoboTwin. To test if this improvements also impacts robotwin full, we train a new XL (2B) run, to try and push the performance of the method even further. If it does, we will then update the RoboTwin full runs by rerunning them in the optimal configuration. As these runs started close to the end of the discussion, we will report the results here if they finish in time, or in the revision otherwise.

Thank you for the additional ablation and matched-task comparison.

However, my main concern remains unresolved and is in fact reinforced by the new ablation. The paper’s central claim is that decoupling the policy architecture from a generative backbone lets architecture and capacity be chosen for control; concretely, the World and Action DiTs are trained from scratch in a pretrained JEPA feature space. Yet none of the current ablations isolates this contribution. The leave-one-out table shows the largest delta comes from replacing V-JEPA features with Wan2.2 VAE latents (−13.7 clean / −26.5 randomized on RoboTwin), followed by future prediction and image augmentation. The architecture-decoupling choice itself is never removed or varied.

I therefore remain concerned that the central architectural claim is not empirically supported, and I will keep my score at 2 for now.

Reviewer_BZJyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper primarily replaces the VAE latents in Fast-WAM with V-JEPA representations and employs a compact policy backbone trained from scratch, while retaining the core training and inference mechanisms of Fast-WAM. Consequently, the methodological innovation is relatively limited.

  1. Representation and Architecture: The Fast-WAM framework is adapted to the V-JEPA 2.1 representation space. The pretrained video generation backbones are replaced by compact, trained-from-scratch World DiT and Action DiT models, while maintaining the core mechanism: joint prediction of future states and actions during training, and action-only generation during inference.
  2. Training and Optimization: The approach incorporates motion encoding, dropout applied to camera and robot states, image augmentation, and LoRA fine-tuning of the visual encoder to enhance the generalization capabilities of the policy.
Critical strengths
  1. Compact architecture and efficient inference: By eliminating the backbone of large generative models and employing a control network trained from scratch, the approach significantly reduces the number of parameters, achieving substantially faster inference than Fast-WAM.
  2. Comprehensive ablation studies: The work provides a systematic analysis of representation space, future prediction, motion encoding, and training strategies, offering both empirical evidence for performance gains and valuable design insights.
Critical weaknesses
  1. Limited methodological innovation: The core framework remains based on Fast-WAM, with the primary changes being the adoption of V-JEPA representations and a compact, trained-from-scratch backbone.
  2. Lack of validation for scalability: Larger models failed to show consistent performance gains on RoboTwin, leaving the architecture’s scalability potential unproven.
Questions for authors

It is recommended to conduct additional experiments on RoboTwin with models containing 2B or more trainable parameters and report the mean and standard deviation of success rates across multiple seeds to assess scalability and result stability.

2 comments

We thank the reviewer for their feedback. We also invite them to take a look at our general response, which reports updated scores on our benchmarks.


W1: We agree that the architecture isn't novel; it isn't the main point of our paper. What is novel here is a policy trained from scratch in a given latent space: the vast majority of WAMs start from heavily pretrained backbones whose generators have seen large amounts of data at heavy cost, yet tailoring the pipeline to robotics directly gives better performance, faster inference and cheaper training. Without a pretrained generator backbone, the policy can be sized for control: La-JEPA trains 0.1–1.1B parameters against 6 and 5B for Fast-WAM and Faster-WAM, runs 2–5× faster, and outperforms them on every tested benchmark except LIBERO (where all methods are near saturation), now including the full RoboTwin.

Please note that our comparisons are also generous to the baselines: they all benefit from large-scale generation pretraining, and the Wan video generator behind Fast-WAM and Faster-WAM was trained on several billions of images and videos for encoding (Wan2.2's 3D causal VAE) and generation (the actual backbone), against 163M for V-JEPA 2.1 trained as an SSL encoder only. This leaves headroom: La-JEPA is already matching or exceeding the state of the art among policies without embodied pretraining, and could further benefit from similar pretraining. However, no pretrained backbone exists in JEPA space, and pretraining one is beyond the scope of this paper.


W2: We push back on the argument that the RoboTwin clean-to-clean and clean-to-randomized numbers show a lack of scalability from the method: the small size of the clean RoboTwin data (about 50 demonstrations per task) is the limiting factor here, and scaling up the model won't help without adding data.

Still, we agree that scalability needs to be shown where the dataset size is adequate, so we train on the full RoboTwin set, about 10× more data (general response §3). We report clean, randomized and mean success (clean-only training: three seeds, ± SD; full training: one seed):

SizeParamsClean-only: cleanClean-only: rand.Clean-only: meanFull: cleanFull: rand.Full: mean
S0.1B85.45 ± 0.4933.87 ± 2.1659.66 ± 1.2989.2089.3889.29
B0.37B86.20 ± 0.5035.43 ± 0.6060.82 ± 0.5092.5691.5092.03
L1.2B85.99 ± 0.4232.07 ± 2.7559.03 ± 1.5893.3492.6693.00

With clean-only data, the three sizes are within seed spread; with full data, success increases monotonically with size, and L matches generator-based WAMs that train 4–5× more parameters. While clean-only randomized success varies by 2 to 3 points across seeds for S and L, all sizes stay well clear of the next best comparable method (mean 59–61 vs. 47.1 for Qwen-RobotManip-scratch). The requested 2B-parameter model is still training, and we will add it as soon as it finishes.

We also provide the means and standard deviations for LIBERO and LIBERO-Plus (mean ± SD over three seeds; LIBERO-Plus with the official text):

SizeLIBEROLIBERO-Plus
S97.93 ± 0.0679.97 ± 0.38
B98.02 ± 0.2082.49 ± 0.44
L98.07 ± 0.2484.07 ± 0.41

W2, Question

As promised, the requested 2B-parameter model (La-JEPA XL, 2.1B) has finished training on the full RoboTwin set. It reaches 94.2 clean and 93.2 randomized (mean 93.7, vs. 93.0 for L): success keeps increasing with size, and XL sets a new state of the art among policies without embodied pretraining (see the update to our general response).

Comments on the paper

2 comments

We thank the reviewers for their feedback. We summarize here the updates that apply to all reviews: updated scores (§1), an issue with the released LIBERO-Plus instructions (§2), and new results on the full RoboTwin training set (§3).


1. Updated evaluation pipelines to match baselines more closely

Headline numbers are now a mean over three training seeds, additional seeds for RoboTwin-full will also be added, some evaluation details have been fixed to more closely match baseline evaluations.

LIBERO. Newer MuJoCo releases displace some objects at reset and change the look of the scenes (darker objects and floors). Our submission used such a release, while the baselines evaluate with older ones, so to be fair to our method we now match their setup (MuJoCo 3.3.2), which significantly raises our scores.

RoboTwin. We now follow the official per-episode instruction protocol instead of one fixed instruction per task (Appendix B). Clean success rises by 5 to 8 points and randomized success falls by 5 to 9, so the mean of the two regimes stays within a point of the submission.

Scores: submission → revised.

SizeLIBEROLIBERO-Plus, official textRoboTwin cleanRoboTwin rand.
S93.4 → 97.975.2 → 80.0 ± 0.480.2 → 85.5 ± 0.539.0 → 33.9 ± 2.2
B96.4 → 98.078.9 → 82.5 ± 0.478.8 → 86.2 ± 0.544.7 → 35.4 ± 0.6
L97.0 → 98.182.3 → 84.1 ± 0.478.4 → 86.0 ± 0.439.1 → 32.1 ± 2.8

Our RoboTwin conclusions don't change: La-JEPA is still the strongest policy without embodied pretraining by a large margin, in both regimes. Our LIBERO-Plus results for B and L can now too claim state of the art among these policies.


2. LIBERO-Plus noisy instructions correction .

For 8,493 of the 10,030 variants, the released LIBERO-Plus evaluator builds the instruction from the variant's file name, so the policy receives the task followed by the perturbation parameters (e.g. "… place it on the plate light 1"). This issue recently came to light (noted by Flex-π), and it leaks the perturbation into the language input. Flex-π replaces the instruction with the BDDL annotation, which still differs from the training instruction for most variants, so we instead restore the exact training instruction. The language variants, which are the real language perturbations, are untouched. We re-evaluate all baselines from their public checkpoints, with both texts and the official step limits on all 10,030 variants. Our pipeline matches independent re-evaluations of the released checkpoints under the same protocol (π0.5: 84.3 vs. 84.4; Cosmos Policy: 80.8 vs. 80.9; IndustrialVLA-Bench [Wang et al., 2026]). Several published scores are higher because they average categories instead of episodes, evaluate a subsample, or use longer step limits (e.g. ImageWAM's reported 83.1 covers a 15% subsample).

PolicyOfficial textClean instructionΔ
Without embodied pretraining
La-JEPA S (3 seeds)80.0 ± 0.486.3 ± 0.6+6.3
La-JEPA B (3 seeds)82.5 ± 0.487.3 ± 1.0+4.9
La-JEPA L (3 seeds)84.1 ± 0.488.0 ± 0.3+3.9
Cosmos Policy80.884.9+4.1
ImageWAM79.780.4+0.7
LeapBot-WA71.779.7+8.1
JEPA-WAM77.177.8+0.7
StarVLA76.777.6+0.9
With embodied pretraining
π0.584.385.7+1.4
π-JEPA-WAM84.585.2+0.6

Bold: best, italics: second best (among policies without embodied pretraining).

With clean instructions, LIBERO-Plus measures what it was designed for: robustness to the perturbation itself. The corrupted text costs policies 0 to 8 points, so official scores partly rank them by their sensitivity to it; we hypothesize this depends on the text encoder, as policies with an LLM or VLM encoder barely change (at most 1.4 points) while those with a T5-family encoder gain 4 to 6. Under the corrected benchmark, La-JEPA S is already SOTA, and scaling pushes the boundary further. We keep the official text in the rest of this rebuttal for comparability with published numbers, and we will add the correction, its code and the re-evaluated baselines as secondary contributions.


3. Scalability experiment, full RoboTwin training set.

To test scaling where data is adequate, to answer Reviewer BZJy's question, we also train on the full RoboTwin set (clean and randomized demonstrations, about 10× more data than the clean-only setting):

ModelTrainable paramsCleanRandomized
La-JEPA S0.1B89.289.4
La-JEPA B0.37B92.691.5
La-JEPA L1.2B93.392.7
Qwen-RobotManip-scratch (reported)4.7B88.788.4
Faster-WAM (reported)5.1B89.788.6
Fast-WAM (reported)6.0B91.991.8
ImageWAM (reported)4.5B93.293.6

Success increases monotonically with size, and L matches generator-based WAMs that train 4–5× more parameters. A 2B-parameter model is currently training and will be added once finished.

Update: 2B-parameter model on the full RoboTwin set. The 2.1B-parameter La-JEPA XL announced in §3 has finished training (one seed per size):

ModelTrainable paramsCleanRandomizedMean
La-JEPA S0.1B89.289.489.3
La-JEPA B0.37B92.691.592.0
La-JEPA L1.2B93.392.793.0
La-JEPA XL2.1B94.293.293.7
Qwen-RobotManip-scratch (reported)4.7B88.788.488.6
Faster-WAM (reported, arXiv v2)5.1B90.490.690.5
Fast-WAM (reported)6.0B91.991.891.8
ImageWAM (reported)4.5B93.293.693.4

Bold: best, italics: second best.

Success keeps increasing with size. XL surpasses ImageWAM, the previous best policy without embodied pretraining, and sets a new state of the art with the best clean and mean success (94.2 and 93.7). To make sure these comparisons are fair, we re-ran the released ImageWAM checkpoint in our pipeline (93.6 clean / 93.2 randomized, vs. 93.2 / 93.6 reported).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.