acceptodds
Under review as a conference paper at ICLR 2027

Video Prediction Policy 2: Predict Better, Act Better

58 views

Abstract

World action models (WAMs) have emerged as an important class of generalist robot policies, aiming to transfer video prediction priors to action learning. However, we find that existing WAMs frequently produce incorrect motion predictions in open-ended environment, leading to erroneous actions. We attribute this limitation to two factors: (1) base video models are not optimized for manipulation, and (2) naively incorporating action components into video models can substantially degrade their generalization capabilities. We introduce Video Prediction Policy 2 (VPP2), a WAM that enables strong zero-shot generalization in both video prediction and action generation. First, we curate a large-scale, diverse dataset of manipulation videos to continue pretraining the base video foundation model. We annotate video clips with detailed captions and perform event-level video pretraining to promote generalization across open-ended manipulation tasks. Second, we post-train and distill the video model into a single-step visual planner with fixed prediction horizon. Finally, we introduce action module via a mixture-of-transformers (MoT) architecture to learn implicit inverse dynamics model. Experiments demonstrate three key results: (1) VPP2-14B outperforms Cosmos3-64B by 11% in instruction following on video prediction in open-ended tasks; (2) VPP2 exhibits strong zero-shot manipulation capabilities on a real-world ALOHA platform; and (3) following benchmark-specific post-training, VPP2 achieves the highest success rates among evaluated methods on the challenging LIBERO-Pro, LIBERO-OOD, and RoboDojo benchmarks. Videos can be found at https://sites.google.com/view/video-prediction-policy-2sites.google.com/view/video-prediction-policy-2.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world action model, video prediction

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_E1wQOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

VPP2 continues pretraining Wan2.1-I2V-14B on captioned manipulation events, then adapts and distills it for single-step fixed-horizon prediction. A newly initialized 0.9B action expert learns from bimanual datasets through a standard mixture-of-transformers interface. The paper evaluates video instruction following, ALOHA deployment without task-specific finetuning, and benchmark-specific post-training on LIBERO and RoboDojo. The contribution is mainly the data and staged training recipe, rather than a new action architecture.

Critical strengths
  1. The staged training recipe is a substantial empirical contribution. Complete-event supervision and detailed captions align instructions with full manipulation trajectories; fixed-horizon adaptation and single-step distillation then make this model usable for action learning. Table 2 supports the event-level design: instruction-following success is 0.80/0.90 on human/robot cases versus 0.48/0.42 for Fixed-Step. This connects a clear training motivation with a large video-level improvement.
  2. Strong policy results across real hardware and compositional benchmarks support the recipe's practical value. ALOHA reaches 58.5% versus 40.0% for the strongest compared policy, with baselines exposed to the same ALOHA action datasets (Fig. 8). After benchmark-specific post-training, LIBERO-Pro reaches 45.0% versus 11.0%, and LIBERO-OOD 63.9% versus 59.4% for the strongest reported baselines (Table 3). These results make the contribution compelling beyond near-ceiling standard LIBERO performance.
Critical weaknesses
  1. The ALOHA evaluation needs a more precise scope. The ten categories do not identify which task instances, objects or compositions were excluded from video and action training. Section 3.3 also describes VLM planning/prompt enhancement without specifying its use for each system in Figure 8. These details affect the interpretation of zero-shot transfer and the comparison. Table 4 explicitly excludes subtask planning, so that ambiguity does not apply to the main RoboDojo result.
  2. The event-level ablation establishes a video-prediction benefit, while its contribution to final policy performance remains less isolated from captioning, data and subsequent adaptation. The complete-system results are strong; a downstream control or existing analysis would make the explanation of which training choices matter more informative.
Questions for authors
  1. What task instances, objects or instruction compositions in Figure 8 are held out from both video and action training?
  2. Is there a matched policy-level event-versus-fixed-step result, or existing analysis showing how the Table 2 advantage carries through fixed-horizon adaptation and distillation?
  3. Do the Figure 8 systems receive the same instructions and VLM planning/prompt enhancement? Please specify the configuration for each.
3 comments

Thank you for your thoughtful questions about our evaluation.


W1, Q1. Which task instances, objects, and instructions were used in the ALOHA evaluation in Figure 8?

Figure 8 summarizes zero-shot performance across ten real-world task categories on ALOHA. To construct individual task instances, we used everyday objects from our surroundings and placed them randomly in the workspace. This tests whether a policy can carry out the requested manipulation as the objects and their positions vary. Figure 10 shows a random sample of the test tasks and gives concrete examples of the objects and arrangements behind the category-level results. For each task, we used Qwen3-VL [1] to expand the original instruction into a more detailed prompt, which we shared across policies.


W2, Q2. How does event-level pretraining affect policy performance?

We varied only how the training videos were segmented in Stage 1 of VPP2 pretraining. A fixed-length clip can end before the action described by its instruction is complete. Using a complete subtask event keeps the instruction paired with the trajectory through the end of the action. We kept all subsequent training stages identical and evaluated both policies in simulation and on the real robot.

Simulation. Both variants achieved near-perfect success on LIBERO-ID (98.6% with fixed-length clips and 98.8% with complete events), leaving little room for improvement on this benchmark. On the more challenging LIBERO-Pro and LIBERO-OOD benchmarks, event-level pretraining improved success by 4.9–13.2 percentage points in four of the five categories. The largest gain was on OOD Goal, where success rose from 50.2% to 87.7% (+37.5 points).

MethodLIBERO-IDPro PositionPro TaskPro OverallOOD SpatialOOD ObjectOOD GoalOOD Overall
VPP2(fixed-length)98.630.734.632.654.434.450.246.3
VPP2(event-level)98.842.347.845.059.344.587.763.9
Gain (pp)+0.2+11.6+13.2+12.4+4.9+10.1+37.5+17.6

Real-world zero-shot evaluation. Given the limited time available during the rebuttal, we evaluated both policies on three real-world task categories (stacking, pick-and-place, and towel folding). Event-level pretraining raised the mean success rate across these three categories from 51.7% to 70.0% (+18.3 percentage points). Stacking and towel folding each improved by 10 points, and pick-and-place showed the largest gain, from 40% to 75%. These results show that the gains observed in simulation also extend to the real-world tasks we tested.

MethodStackingPick-and-placeTowel foldingAverage
VPP2(fixed-length)55.040.060.051.7
VPP2(event-level)65.075.070.070.0
Gain (pp)+10.0+35.0+10.0+18.3

Q3. Did all policies in Figure 8 receive the same instructions and VLM prompt enhancement?

Yes. We used Qwen3-VL to expand each task instruction and supplied the same resulting prompt to all three policies (π0.5\pi_{0.5}, Fast-WAM-14B, and VPP2). We followed this procedure during both training and evaluation. In particular, any task details added during prompt enhancement were available to all three policies at test time, so the comparison in Figure 8 holds the language input constant.

[1] Qwen3-VL Technical Report. arXiv:2511.21631, 2025.


Thank you again for your constructive comments and support. We hope these responses address your concerns, and we would be happy to clarify any additional questions.

Reviewer_E1wQOpenReview ↗

Response to authors

Thank you for the response. It addresses my questions, and I’m keeping my rating at 4.

Thank you for your constructive comment and support on our paper!

Reviewer_rxKtOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes VPP2, a world action model that aims to improve generalization through better instruction conditioned video prediction. The main contribution is a large scale manipulation video training pipeline that segments demonstrations into semantic events and annotates them with detailed descriptions of object interactions, active end effectors, and target objects. The authors use this data for event level video pretraining, followed by fixed horizon post-training and consistency distillation to obtain a single-step visual planner. An action expert then generates robot actions conditioned on the predicted video representations. The authors evaluate video instruction following, zero-shot task generalization on ALOHA, and performance on LIBERO-Pro, LIBERO-OOD, and RoboDojo, reporting strong improvements over existing methods.

Critical strengths
  1. The event level data curation and language grounding pipeline is a key strength. Aligning detailed captions with complete manipulation events helps reduce ambiguity in video supervision, and scaling this across diverse human and robot datasets is a useful contribution.
  2. The experimental results are compelling, particularly on LIBERO-Pro and LIBERO-OOD. VPP2 compares favorably with existing WAM baselines on LIBERO-OOD and RoboDojo, while the real world ALOHA results provide further evidence of task generalization.
  3. The paper presents a thoughtful and well-engineered training pipeline, progressing from manipulation video pretraining to fixed-horizon prediction and efficient action generation. While the individual components are largely established techniques, their integration into a practical and effective system is a worthwhile contribution. The released model checkpoints and implementation code add to its practical value.
Critical weaknesses
  1. The contribution of event-level pretraining to downstream policy performance is not sufficiently established. The VPP2-Fixed-Step ablation demonstrates the benefit of event-level training for video instruction following, which supports the authors' motivation. However, it remains unclear whether these improvements translate to better robot control. A useful ablation would compare policies initialized from event-level and fixed-step pretrained video models, keeping subsequent post-training, distillation, and action training matched. Evaluating both on LIBERO-Pro/OOD would help establish whether event-level pretraining improves downstream generalization.
  2. The benefit of explicit future prediction remains unclear. Fast-WAM (Yuan et al., 2026) suggests that much of the benefit of video modeling may come from learning stronger representations rather than generating future observations at inference. While VPP2 demonstrates strong downstream performance, it does not isolate the contribution of explicit future prediction. A comparison using current observation features from the same pretrained video backbone, with matched action architecture and training, would help clarify this.
  3. The methodological novelty is somewhat limited. The key components, including event-level supervision, video model adaptation, and video-conditioned action generation, have precedents in prior work. The contribution lies primarily in the large scale data curation and thoughtful integration of these techniques into an effective policy, rather than a fundamentally new approach.
Questions for authors
  1. Do the advantages of event-level video pretraining translate into downstream policy improvements? A comparison using event-level and fixed-step pretrained video models, with matched subsequent training conditions, would help establish this.
  2. How much does explicit future video prediction contribute beyond the pretrained video representations themselves? Is there a comparison using current observation features from the same backbone, with matched action architecture and training?
  3. The event segmentation and dense annotation pipeline appears to be a valuable contribution in its own right. Do the authors plan to release the event boundaries, detailed captions, and data processing pipeline? Making these annotations publicly available, even independently of the underlying videos, could substantially benefit the community.
5 comments

We sincerely appreciate your time and effort in reviewing our paper.


W1, Q1: The contribution of event-level pretraining to downstream policy performance.

We compare two VPP2 policies that use either fixed-length video clips or complete subtask events for Stage 1 pretraining. All subsequent training stages are identical.

Simulation. Both policies are near ceiling on LIBERO-ID (98.6% and 98.8%). Event-level pretraining improves success by 4.9–13.2 percentage points in four of the five LIBERO-Pro and LIBERO-OOD categories and by 37.5 points on OOD Goal.

MethodLIBERO-IDPro PositionPro TaskPro OverallOOD SpatialOOD ObjectOOD GoalOOD Overall
VPP2(fixed-length)98.630.734.632.654.434.450.246.3
VPP2(event-level)98.842.347.845.059.344.587.763.9
Gain (pp)+0.2+11.6+13.2+12.4+4.9+10.1+37.5+17.6

Real-world zero-shot evaluation. Given the limited time available during the rebuttal, we evaluated both policies on three real-world task categories (stacking, pick-and-place, and towel folding). Event-level pretraining improves success in all three task categories. The mean success rate across categories rises from 51.7% to 70.0% (+18.3 percentage points), with the largest gain on pick-and-place (+35.0 points).

MethodStackingPick-and-placeTowel foldingAverage
VPP2(fixed-length)55.040.060.051.7
VPP2(event-level)65.075.070.070.0
Gain (pp)+10.0+35.0+10.0+18.3

W2, Q2: What does explicit future prediction add?

Thank you for the insightful comment. Actually, we already trained the Fast-WAM baseline in Figure 8. For a fair comparison, we use the same ALOHA manipulation data and Wan2.1-I2V-14B backbone. The only difference at inference is that VPP2's action expert also receives representations of predicted future observations. Across 10 real-world zero-shot task categories, average success increased from 20.5% with Fast-WAM to 58.5% with VPP2. This 38-point gain suggests that predicted future representations help guide the robot arm's actions in unfamiliar settings.


W3, Q3: About novelty and data processing pipeline.

As you mentioned in the Strength part, We agree that our contribution lies in how the components work together. We designed the full data processing pipeline and training objectives together to preserve the pretrained video model's ability to generalize when adapted for robot control. The pipeline pairs complete manipulation events with captions that specify the acting arm and target object. Training then progresses from event-level video prediction to fixed-horizon prediction and finally to action learning, with lightweight updates to the video backbone in the final stage. Further, We will release the processing and training code along with model checkpoints upon acceptance.


Thank you again for your constructive comments and support. We hope these responses address your concerns, and we would be happy to clarify any additional questions.

Thank you for the rebuttal and additional experiments. The new ablation provides further evidence that event-level pretraining improves downstream policy performance and addresses my main concern. The Fast-WAM comparison is useful, but a more controlled ablation would be a VPP2 variant that uses the same pretrained video backbone while conditioning the action expert only on current observations, without future prediction. I realize this may be difficult to address within the rebuttal window, but I would encourage the authors to include this comparison in the final version. Also, do the authors plan to release the generated event boundaries and dense captions for the publicly available datasets, in addition to the processing code?

We appreciate the follow-up and are glad the new ablation addressed your main concern about event-level pretraining. The entry labeled “Fast-WAM” in Fig. 8 is exactly the current-observation-only VPP2 variant you suggest. It uses the same pretrained video checkpoint, action-expert architecture, and training data as VPP2. The only change is the attention mask: the action expert can attend to tokens representing the current observation, but has no access to the tokens representing predicted future observations. Across the ten real-world task categories in Fig. 8, this variant achieved a mean success rate of 20.5%, compared with 58.5% for VPP2. In the final version, we will label this variant “VPP2 (current observation only)” and explain the masking setup in the figure caption.

We will release the data-processing code upon acceptance. We also aim to share the generated event boundaries and dense captions for the publicly available datasets. We have not yet finalized which datasets we can include, since we need to check each source dataset’s terms for redistributing derived annotations. We will specify the coverage dataset by dataset when we release the code.

Thank you again for your time and effort in reviewing our paper!

Reviewer_yTUYOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This work presents Video Prediction Policy 2, a new robot control policy. The policy itself is a World Action Model, i.e., a post-trained video prediction model. The key problem that the work addresses is a lack of generalization in WAMs, which it attributes to a lack of knowledge of precise physical dynamics and generalization degradation as a consequence of action-specific fine-tuning. It presents a pipeline to mitigate these issues and evaluates the work on several standard robotics benchmarks and an own dataset.

Critical strengths

The work's main strength is its empirical performance, as it outperforms many baselines.

Despite some missing parts, the writing was very clear, and the authors clearly cared about their work and did a great job at polishing.

Critical weaknesses

The work struggles to clarify what the reader learns from it. This isn't necessarily bad, since it is a systems contribution presenting an overall system that achieves state-of-the-art performance, so it shouldn't necessarily be held against the authors. Even for systems-level work, the authors need to clarify which design choices were obvious, what was challenging, and what new insight led to this performance. I.e., the work needs to tell its readers why it is interesting from a research perspective.

As it stands, the work is unreproducible, which is a deal-breaker. I double-checked this with an LLM and will list its findings below verbatim. I tried to filter the incorrect ones out and double check as good as I could. Even after checking it still seems that numerous points are missing for the work to be reproducible.

This is less of a critical weakness but more of a point for potential improvement. While the writing is very clear, it is largely only understandable to researchers who already work in robot learning. Even basic knowledge in either Computer Vision or Robotics isn't enough to understand this paper, and it is completely out of reach for the average ICLR attendee, where the common knowledge denominator assumption is a grad-level deep learning/ML course.

LLM output on missing information for reproducibility

Data

  • Corpus size is missing: clips, hours, or frames per source in Table 1, the total, and the mixture or sampling weights across sources.
  • The self-collected ALOHA, dexterous-hand, and egocentric data have no stated amount, tasks, hardware, or cameras.
  • The per-dataset filtering strategies (Table 1 caption) and the general-video filter ("similar to LVP") are not specified: criteria, models, and thresholds are all missing.
  • For segmentation, the paper doesn't give:
    • the velocity and gripper-event thresholds
    • how hand velocity is obtained for human videos
    • which VLM refines the segments, and its prompt
  • For captioning, the paper doesn't give:
    • the VLM identity and version
    • the full caption template or prompt
    • the optical-flow method and threshold for discarding egocentric clips
  • The unified action space is missing:
    • the action parameterization: absolute vs. delta end-effector pose, rotation representation, gripper/hand encoding
    • action dimensionality, normalization, and control frequency
    • whether proprioception is an input
    • how dexterous-hand, single-arm, human-hand, and non-egocentric (~20%) data are mapped or excluded

Stage 1 (event-level pretraining)

  • The optimizer and its settings are missing: LR schedule and warmup, weight decay, gradient clipping, EMA, and precision.
  • The flow-time sampling distribution (uniform, logit-normal, or shifted) is not stated.
  • The text-dropout rate for CFG is not stated.
  • The paper doesn't say which components are trainable, e.g. whether the text encoder, CLIP image encoder, and VAE are frozen.
  • There's no description of how events shorter than N=16 frames are handled, or how differing source frame rates are handled.

Stage 2 (chunk post-training and consistency distillation)

  • Post-training steps, batch size, LR, and data mixture are not given.
  • For consistency distillation, the paper doesn't give:
    • the weighting λ(s, s′)
    • how F_θ is parameterized in terms of v_θ
    • the teacher ODE solver and step count used to compute x̂_{s′}
    • the sampling distributions for s and for s′ < 1
    • the teacher CFG scale and the EMA decay
    • steps, batch size, LR, and student initialization
  • The multi-step sampler and CFG scale behind the 30-step results in Fig. 5 are not stated.

Stage 3 (action expert)

  • The 0.9B DiT architecture is undescribed: depth, width, heads, and how its layers pair with Wan's blocks in the MoT.
  • The action chunk length in steps is missing (only "2 seconds" is given), as are the action tokenization, action loss, and noise schedule.
  • It's unclear whether the video loss is kept during Stage 3, and what the loss weights are.
  • It's unclear which video latents condition the actions during training: ground truth, noised, or one-step distilled outputs. This matters because inference uses the one-step output.
  • The freeze/LoRA schedule is missing:
    • how long the backbone stays frozen
    • LoRA rank, alpha, and target modules
    • whether full unfreezing follows
    • total steps, batch size, and LR
  • Which datasets enter action training, and in what mixture, is not stated.
  • Deployment details are missing: actions executed per replanning step, control rate, and the GPU used for the 0.22 s latency figure.

VLM planner and prompt enhancement

  • The paper doesn't say which VLM is used in each experiment (both Qwen3-VL and Seed2.0 are cited).
  • The planning and prompt-enhancement prompts are not given, nor how "execution feedback" and subtask completion are determined.
  • It's unclear whether prompt enhancement or planning was used in Table 2, the ALOHA experiments, and LIBERO, and whether baselines received the same enhanced prompts.

Evaluation

  • Video prediction:
    • The 50+50 image–instruction pairs are not released, including the authors' own photos.
    • The GPT evaluator's model version, prompt, success criterion, and the frames it was shown are not given.
    • The number of samples or seeds per pair is not stated.
    • The human-evaluation protocol is missing: number of raters, blinding, and how ties were decided.
    • Baseline generation settings (steps, CFG, resolution) are not given.
    • It's unclear which VPP2 checkpoint and step count were used, and how VPP2-Fixed-Step was trained.
  • ALOHA zero-shot:
    • The task list, exact instructions, and scene setups are not given.
    • Trials per task are not stated (the 5% granularity suggests 20), nor are the success criteria.
    • The paper doesn't explain how the tasks were verified as unseen.
    • The fine-tuning recipes for π0.5 and Fast-WAM are missing, including their starting checkpoints.
  • LIBERO:
    • The fine-tuning recipe is missing: starting checkpoint, steps, LR, LoRA vs. full fine-tuning.
    • There's no explanation of how single-arm Franka actions map into the bimanual action space, or which camera views are used.
    • Episode counts for LIBERO-ID, LIBERO-Pro, and LIBERO-OOD are not given.
  • RoboDojo:
    • The fine-tuning recipe is missing.
    • For subtask planning, the demonstration segmentation and the VLM used are not described.
    • VPP2's generalization score is reported only pooled, while baselines are split into Std./Rand.
  • No variance across seeds or confidence intervals is reported for any result.

Inconsistencies worth raising

  • Video-model latency is about 0.1 s in the Fig. 4 caption but about 0.12 s in Sec. 3.2.
  • Fig. 4 depicts joint video-and-action denoising in Stage 3, while Sec. 3 says the action expert conditions on the video model's KV cache. These imply different training procedures.
Questions for authors
  • In the intro, you claim that the fact that "base video models are primarily optimized for creative content generation and aesthetic quality rather than precise physical dynamics" is a key factor that limits generalization. I don't understand this argument. Aren't these two things orthogonal? I.e., you could also have a model that fails to generalize even if the physical dynamics are correct, but the appearance is poor. In the end, it seems to depend on the nature of generalization.

  • Will the authors release the dataset that they collected?

5 comments

We sincerely appreciate your time and effort in reviewing our paper. We provide detailed responses below:


Weakness 1: About contribution and reproducibility

We have included as much implementation detail as space allows. To support replication, we will release the codebase and model checkpoints upon acceptance. Below, we clarify the design choices and evaluation protocol raised in the review.

Contribution. We designed the data pipeline and training objectives together to preserve the video model's ability to generalize when adapted for robot control. The policy learns from complete manipulation events and uses predicted future observations to guide its actions.

How can the method and evaluation be reproduced? During Stage 3 training, the video model starts at the maximum noise level and predicts a future video in one denoising step. The action expert reads the resulting representation to generate the robot's actions. We use the same conditioning procedure at inference. Section 3.2 breaks down the latency per action chunk into approximately 0.12 s for video prediction and 0.10 s for action generation, or 0.22 s in total.

For Figure 8, we randomly sampled 20 trials within each of the ten ALOHA task categories. All three policies received the same enhanced instruction for a given task. We calculated each category's success rate as successful trials divided by 20 and reported the mean of the ten category rates.


Q1: About the claim that base model are good at creative content but not precise physical dynamics

Prior studies have also found that generated manipulation videos can look plausible while depicting physically inconsistent scenes. LVP [1] (Fig. 5) reports spatial errors in hand-object interactions, and Qwen-RobotWorld [2] (Sec. 1) describes objects changing shape across frames. In our own evaluation, the base Wan2.1-I2V-14B model sometimes introduced objects that were absent from the initial scene and were not called for by the instruction. We will revise the text to describe this specific failure mode.

[1] Large Video Planner Enables Generalizable Robot Control. arXiv:2512.15840, 2025.

[2] Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation. arXiv:2606.17030, 2026.


Q2: About the dataset we collected.

Most of our training data come from publicly available datasets that we use in accordance with their respective terms. The data we collected ourselves still require privacy review and de-identification before they can be shared. We therefore do not expect to release this portion in the near term.


We hope these responses address your concerns, and we would be happy to clarify any additional questions.

We further clarify several key details below. To provide more detailed information and support replication, we will release the full codebase and model checkpoints upon acceptance.

Benchmark adaptation data

For LIBERO, we train on the four standard suites using 1,712 demonstrations and 277,713 frames after no-op filtering: Spatial (434 demonstrations), Object (457), Goal (433), and Long (388). Sampling across suites is proportional to their frame counts. We use all demonstrations for training and perform no additional training on LIBERO-Pro or LIBERO-OOD.

For the RoboDojo, the dataset contains 3,500 demonstrations and 1,856,102 frames at 25 Hz.

LIBERO and RoboDojo evaluation protocols

BenchmarkEvaluation protocolEpisodes
LIBERO-ID40 tasks × 50 packaged initial states2,000
LIBERO-OOD30 tasks × 3 deterministic reset-seed IDs × 50 random resets4,500
LIBERO-Pro40 tasks × 2 perturbations (Position and Task) × 10 initial states800

LIBERO-ID uses initial-state indices 0–49, and LIBERO-Pro uses indices 1–10. The OOD seed IDs specify evaluation resets. LIBERO evaluation uses the benchmark task instruction with a fixed robot-view prompt wrapper, without VLM prompt enhancement or a high-level planner.

For RoboDojo, we are following the official leaderboard evaluation protocol.

Trainable components

The text encoder, CLIP image encoder, and VAE remain frozen throughout training; their pretrained weights are not updated.

Robodojo Generalization results

VPP2 achieves scores of 46.03 / 18.65 and success rates of 39.33% / 10.67% under the Std. / Rand. settings, respectively. We will include these separate results in the corresponding appendix tables in the final version.

We hope these responses address your concerns, and we would be happy to clarify any additional questions. As the rebuttal period draws to a close, we look forward to your reply.

Most of our training data come from publicly available datasets that we use in accordance with their respective terms. The data we collected ourselves still require privacy review and de-identification before they can be shared. We therefore do not expect to release this portion in the near term.

That can be fine. In such a case I think it is important that the information about data collection and type of data provides enough detail such that someone can reproduce the results by collecting their own dataset following the information provided in the paper.

Comments on the paper

1 comment
Area_Chair_vtC2OpenReview ↗

Discussion with the authors

Dear reviewers,

the authors have provided responses for your reviews. Can you please let them know if your comments have been addressed?

Thank you, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.