acceptodds
Under review as a conference paper at ICLR 2027

Pelican-Sim 1.0: A General World Model Simulator for Embodied Intelligence

Abstract

Robot world models must predict not merely plausible videos, but outcomes faithful to specified actions. Across heterogeneous robots, numerical configurations preserve joint-level detail but leave visual consequences implicit, whereas projected visual actions expose motion geometry but obscure depth and configuration information. Our key idea is to let these complementary representations interact throughout generation. We introduce Pelican-Sim 1.0, a shared action-conditioned world model simulator that injects numerical configurations and camera-aligned whole-arm skeleton videos into alternating video-transformer blocks. Each condition shapes hidden states already informed by the other, coupling configuration detail with projected geometry. Sparse mixture-of-experts layers accommodate heterogeneous dynamics, while causal adaptation and few-step distillation support efficient autoregressive rollouts. Trained on approximately one million real and simulated trajectories, Pelican-Sim 1.0 outperforms all evaluated baselines on five video-quality metrics across held-out splits of AgiBotWorld Beta, RoboMIND, and RoboTwin. Ablations demonstrate improvements over single-condition, concatenation, and late-fusion alternatives, supporting the importance of how complementary conditions interact. On RoboTwin, generated-data augmentation raises policy success from 70.0% to 93.0%, while imagined policy optimization with task-specific simulator adaptation improves success from 62.7% to 75.4%. These results support interleaved numerical–visual conditioning as an effective design for shared robot world models and demonstrate their utility for policy learning. We will release model checkpoints and code.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, robot learning, cross-embodiment learning, action-conditioned video generation, mixture of experts, model-based policy optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_KNtoOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Pelican-Sim 1.0, a unified action-conditioned video world model for embodied intelligence. The method combines numerical robot configurations with camera-aligned whole-arm skeleton videos through interleaved conditioning in a video diffusion transformer. It further incorporates sparse MoE layers and few-step autoregressive distillation for efficient rollouts. Experiments demonstrate strong video prediction quality across multiple robot datasets and promising downstream results in data augmentation, policy evaluation, action selection, and policy optimization.

Critical strengths
  1. The combination of numerical configurations and visual action representations is well motivated, and the proposed framework provides a unified approach to modeling different robot embodiments.
  2. The experimental results are strong across multiple datasets and evaluation metrics. The authors also provide useful ablations on action representations, conditioning strategies, and model capacity.
  3. The paper goes beyond video generation metrics by evaluating downstream applications, including policy learning and action selection, demonstrating the potential utility of the learned world model.
Critical weaknesses
  1. The proposed visual action representation is closely related to Visual Action Prompts (VAP), while combining numerical and visual action conditions has also been explored in EnerVerse-AC and ViPSim. Although Pelican-Sim introduces interleaved conditioning, the extent of its architectural advantage over these prior approaches remains insufficiently isolated.
  2. Although the baselines are retrained on the same dataset, they use different action representations. Pelican-Sim benefits from explicit whole-arm geometry derived from URDF and camera metadata, which may provide richer information than the conditions available to some baselines. This makes it difficult to isolate the architectural improvements from the benefits of additional geometric information.
  3. The proposed method relies on accurate URDF models, joint configurations, and camera calibration to construct visual action conditions. Although the paper describes data cleaning and alignment correction, its robustness to noisy or inaccurate geometric parameters remains unclear.
  4. Although the paper reports strong video-quality and motion-related metrics, these measurements do not fully establish the physical accuracy of the predicted interactions. In particular, quantitative evaluations of object motion, contact consistency, and accumulated errors during long-horizon closed-loop rollouts remain limited.
Questions for authors
  1. Can the authors more clearly distinguish the technical contributions from VAP and ViPSim, and provide controlled comparisons under matched action representations and backbones?
  2. What proportion of the world-model training data consists of failure trajectories? Can the authors evaluate failure prediction accuracy, particularly whether the model incorrectly generates successful outcomes for unsuccessful action sequences?
  3. Can the authors compare against baselines equipped with the same visual action conditions, or otherwise control for the amount of geometric information provided?
  4. Can the authors evaluate sensitivity to perturbations in camera calibration, joint states, and URDF parameters?
8 comments
AuthorsOpenReview ↗

Official Comment by Authors

Question 1: Technical distinction from VAP and ViPSim

Controlled comparisons support both explicit whole-arm geometry and the hierarchical interaction design: Pelican retains a quality gain when action representations and the backbone are matched. Tables R7–R8 separate native method characteristics from results obtained after adapting conditioning mechanisms to common inputs and a common backbone.

VAP uses visual skeleton prompts without a separate numerical action branch. ViPSim already combines numerical and visual conditions through visual latent fusion and numerical cross-attention. Pelican-Sim focuses on hierarchical interaction between joint-level numerical configurations and whole-arm visual geometry: synchronized skeletons explicitly represent intermediate-joint and link motion, while numerical scale-shift modulation and visual residual updates operate on shared hidden states. Interleaving is the default schedule.

Table R7. Native method characteristics and conditioning inputs.

MethodConditioning mechanismNumerical conditionVisual action representation
VAPVisual conditioningNo separate numerical branchVisual skeleton
ViPSimVisual latent fusion and numerical cross-attentionEnd-effector poses and camera parametersEnd-effector projections, depth, and morphology
Pelican-SimInterleaved scale-shift and visual residual updatesJoint-level configurationsWhole-arm skeleton

For the controlled comparisons in Table R8, we fix the Cosmos backbone and initialization, MoE configuration, data, training budget, and generation settings. Both visual-only variants disable numerical conditioning; B1–B4 receive the same 28-dimensional joint configuration. Other observations are unchanged.

With geometry matched, Pelican gains 1.1 dB and reduces FVD by 1.8 for end-effector maps (B1→B3), and gains 1.3 dB and reduces FVD by 4.3 for whole-arm skeletons (B2→B4). Thus, richer visual inputs alone do not explain the conditioning advantage.

With injection fixed, whole-arm skeletons improve PSNR by 0.8/1.0 dB and reduce FVD by 1.6/4.1 under ViPSim-style/Pelican conditioning (B1→B2/B3→B4). Since all four variants already receive the same joint configurations, this tests the value of explicitly rendering whole-arm geometry.

With visual inputs matched and numerical branches disabled, Pelican raises PSNR by 0.9 dB and reduces FVD by 2.1 relative to VAP-style (A1→A2). Adding numerical conditioning to Pelican further improves PSNR by 1.0 dB and reduces FVD by 3.3 (A2→B4), supporting complementary numerical and visual information.

Table R8. Controlled comparison results on AgiBotWorld Beta.

MethodPSNR ↑ (dB)FVD ↓
A1: VAP-style (visual only, whole-arm skeleton)20.316.4
A2: Pelican (visual only, whole-arm skeleton)21.214.3
B1: ViPSim-style (numerical + end-effector maps)20.116.9
B2: ViPSim-style (numerical + whole-arm skeleton)20.915.3
B3: Pelican (numerical + end-effector maps)21.215.1
B4: Pelican (numerical + whole-arm skeleton, full model)22.211.0

Note: “Style” denotes adaptation of a conditioning mechanism to the shared backbone and inputs, not the complete original system. In particular, ViPSim-style receives the common 28-dimensional joint input. Full and visual-only Pelican use the same training/evaluation protocol. These are independent controlled re-evaluation results, reported to one decimal place, distinct from the main-table 22.276 dB/10.85 FVD results.

AuthorsOpenReview ↗

Official Comment by Authors

Question 2: Failure trajectories and false-success predictions

Failure trajectories constitute 0% of Pelican-Sim's large-scale world-model pretraining data. The 32.3% failure fraction refers to a separate evaluation pool. On that pool, Pelican-Sim has a 5.73% false-success rate, compared with 37.77% for EnerVerse-AC, a reduction of 32.04 percentage points (Table R9).

We collected 2,000 simulated executions across five RoboTwin tasks using π0.5\pi_{0.5} policy checkpoints from different training steps: 1,354 successes and 646 failures. For each initial state and action sequence, RoboTwin supplies the ground-truth outcome label, while a VLM judges success or failure in the generated world-model rollout. We compare these labels trajectory by trajectory on the same evaluation set.

Among the 646 simulator failures, 37 Pelican-Sim rollouts and 244 EnerVerse-AC rollouts are judged successful. Pelican-Sim also improves success recall by 13.81 percentage points and balanced accuracy by 22.93 percentage points, as summarized in Table R9. This indicates more faithful prediction of simulator outcomes within the evaluated world-model-plus-VLM pipeline, while identifying a remaining false-success rate of 5.73%.

Table R9. Failure prediction against RoboTwin outcome labels.

ModelFalse-success rate ↓Failure recall ↑Success recall ↑Balanced accuracy ↑Overall accuracy ↑
Pelican-Sim5.73%94.27%99.41%96.84%97.75%
EnerVerse-AC37.77%62.23%85.60%73.91%78.05%

Note: Metrics are pooled over the same 2,000 paired executions. False-success rate is the fraction of the 646 simulator failures whose generated rollouts the VLM judges successful. Balanced accuracy is the mean of success and failure recall; overall accuracy uses all 2,000 trajectories. The metrics assess the combined world-model and VLM evaluation pipeline.

AuthorsOpenReview ↗

Official Comment by Authors

Question 3: Controlling the geometric information in action conditions

The added controls show that Pelican's hierarchical conditioning retains a benefit with identical geometric inputs, while explicit whole-arm geometry also contributes to prediction quality. In Table R10, Pelican improves PSNR by 1.3 dB and reduces FVD by 4.3 with identical whole-arm skeletons and numerical joint inputs (Groups 2→4). With identical end-effector maps, it improves PSNR by 1.1 dB and reduces FVD by 1.8 (Groups 1→3). Richer visual inputs alone therefore do not explain the observed advantage.

Explicit whole-arm geometry provides a further gain. With injection fixed, replacing end-effector maps with whole-arm skeletons improves PSNR by 0.8/1.0 dB and reduces FVD by 1.6/4.1 under ViPSim-style/Pelican conditioning. Every group receives the same 28-dimensional joint configuration, so this comparison tests the value of explicitly representing intermediate joints and links.

Original Table 1 compares systems using their native conditioning interfaces. The new controls fix the Cosmos backbone and initialization, MoE configuration, 28-dimensional joint input, data, training budget, and generation settings. Only the visual action representation and injection mechanism change; other observation conditions remain the same.

Table R10. Matched-geometry controls on AgiBotWorld Beta.

GroupVisual action representationInjection mechanismPSNR ↑ (dB)FVD ↓
1End-effector projection mapsViPSim-style adaptation20.116.9
2Whole-arm skeletonViPSim-style adaptation20.915.3
3End-effector projection mapsPelican interleaving21.215.1
4Whole-arm skeletonPelican interleaving22.211.0

Note: These are the same independent controlled re-evaluation results as B1–B4 in Table R8. ViPSim-style adapts the conditioning mechanism to the common backbone and inputs, rather than reproducing the full original system. The comparisons separately support the representation and interaction design within this controlled setting.

Reviewer_peeZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Pelican-Sim 1.0, an action-conditioned video world model for heterogeneous robots. Its main design combines a canonical 28-dimensional numerical configuration representation with camera-aligned URDF-rendered whole-arm skeleton videos, injecting the two conditions in alternating blocks of a Video DiT and using sparse MoE feed-forward layers.

Critical strengths
  • The proposed method achieves impressive results across three benchmarks, including AgiBotWorld Beta, RoboMIND, and RoboTwin, outperforming the evaluated baselines on multiple video quality metrics.
  • The paper provides extensive experiments investigating different action representations, conditioning strategies, and model architectures. The additional qualitative evaluations under changes in robot embodiments, scenes, objects, and viewpoints further illustrate the potential generalization capabilities of the model.
Critical weaknesses
  • The contribution of model capacity and data scale requires further clarification. The improvements from sparse MoE are promising, but the comparison with the dense backbone also changes model capacity. A comparison with a similarly sized dense model would better support the effectiveness of sparse MoE.
  • Most evaluation metrics focus on visual similarity and motion consistency, which do not necessarily reflect the correctness of object dynamics and contact interactions. Since the model is intended to serve as a robot simulator, I believe more direct evaluation of physical outcomes, such as object displacement, grasp success, contact events, and interaction failures, would strengthen the paper.
Questions for authors
  • Could the authors compare sparse MoE against a capacity-matched dense backbone to better isolate the benefits of expert routing?
  • Could the authors provide quantitative evaluations of cross-embodiment generalization and long-horizon action fidelity? In particular, how well does the distilled autoregressive model preserve action consistency over extended rollouts compared with the original model?
  • How do the authors validate that Pelican-Sim accurately models real-world physical state transitions rather than merely generating visually plausible videos? Could the authors provide quantitative evaluations of object motion and contact outcomes against real robot executions, particularly under different action sequences from the same initial state?
8 comments
AuthorsOpenReview ↗

Official Comment by Authors

Question 1: Capacity-matched dense backbone and routing control

Thank you for suggesting a capacity-matched dense comparison. We added Dense-P, a dense backbone matched to MoE in total parameters, and a token-independent routing control. The results show that the MoE advantage persists at comparable capacity, while learned routing gives an additional gain with the expert structure and activation budget fixed.

Matched capacity. In Table R3, Dense-P and learned-routing MoE differ in total parameters by less than 0.005%. MoE improves PSNR by 0.819 dB, reduces FVD by 4.50, and uses 36.77% fewer forward FLOPs. Thus, the observed advantage cannot be attributed solely to a larger total parameter count. This comparison still includes the effects of both sparse structure and routing.

Matched expert structure. The two MoE variants use the same expert bank, top-2 activation budget, shared expert, action conditions, and training protocol. Their parameter counts are identical and forward FLOPs differ by less than 0.005%. Learned routing improves PSNR by 0.323 dB and reduces FVD by 2.01 over token-independent routing, providing more direct evidence for input-dependent expert selection.

Dense-P replaces the RGB backbone's MoE FFNs with dense FFNs of intermediate width 73,728; the remaining structure is unchanged. All comparisons match action conditions, data, training budget, and generation settings.

Table R3. Capacity-matched dense and routing comparisons on AgiBotWorld Beta.

ModelParameters (B)GFLOPs/forwardPSNR ↑ (dB)FVD ↓
Dense-P10.8551882,905.9321.45715.35
MoE: token-independent routing10.8555558,206.4021.95312.86
MoE: learned routing10.8555558,232.8322.27610.85

Note: Parameters include all experts. GFLOPs count major matrix and attention operations per DiT forward for 93 frames at 480×640480 \times 640, batch size 1, and two FLOPs per MAC; text encoding, VAE operations, and sampling overhead are excluded. FLOPs alone do not establish a latency advantage.

AuthorsOpenReview ↗

Official Comment by Authors

Question 2: Cross-embodiment generalization and long-horizon action fidelity

Thank you for requesting quantitative evaluations of generalization and long-horizon fidelity. The added tests show more accurate cross-embodiment prediction than Ctrl-World and quantify the fidelity cost of distillation over 30-second rollouts. At 30 seconds, the four-step student reaches 1.60% whole-arm nKPE, 3.50% object FDE, and 93.00% outcome BA, with a 5.06×5.06\times speedup over the original model.

Cross-embodiment generalization. Table R4 evaluates the four-step model on Sawyer, IIWA, and UR5e, which were excluded from Pelican-Sim world-model training, without target-embodiment fine-tuning. Relative to Ctrl-World, Pelican-Sim reduces whole-arm nKPE by 77.2–80.0 percentage points and object ADE by 77.2–83.8 percentage points, and improves BA by 19.6–20.9 percentage points. These are comparisons of the evaluated systems, rather than an isolation of architecture from pretraining exposure.

Table R4. Cross-embodiment prediction against Ctrl-World.

RobotMethodTest trajectoriesWhole-arm nKPE ↓Object ADE ↓Outcome BA ↑
SawyerCtrl-World10079.3%82.1%63.5%
SawyerPelican-Sim (4 steps)1002.1%3.2%84.0%
IIWACtrl-World10082.4%87.5%62.4%
IIWAPelican-Sim (4 steps)1002.4%3.7%82.0%
UR5eCtrl-World10080.3%80.1%65.1%
UR5ePelican-Sim (4 steps)1001.9%2.9%86.0%

Note: Methods use the same target-robot reference trajectories. nKPE is whole-arm keypoint error; ADE is object average displacement error. BA is balanced accuracy averaged across tasks and repeated runs, not a single pooled 100-trajectory confusion matrix. Spatial errors in Tables R4–R5 are normalized by the image diagonal.

Long-horizon action fidelity. Table R5 compares 30-second RoboTwin rollouts. Relative to the original bidirectional model, the student's 30-second nKPE and object FDE increase by 0.30 and 0.50 percentage points, while BA falls by 2.00 percentage points. Generation takes 48 instead of 243 seconds, a 5.06×5.06\times speedup (80.25% less time). The nKPE gap grows from 0.10 percentage points at 6 seconds to 0.30 at 30 seconds, indicating accumulated prediction error. Relative to the causal teacher, the student's 30-second nKPE increases by 0.20 percentage points and BA falls by 1.50 percentage points.

Table R5. Action fidelity and generation time over 30-second rollouts.

ModelStepsnKPE @ 6 s ↓nKPE @ 18 s ↓nKPE @ 30 s ↓Object FDE @ 30 s ↓BA @ 30 s ↑Time for 30 s video ↓
Original bidirectional model350.70%1.00%1.30%3.00%95.00%243 s
Causal teacher350.70%1.10%1.40%3.10%94.50%212 s
Distilled autoregressive model40.80%1.20%1.60%3.50%93.00%48 s

Note: Models use the same initial observations, fixed action sequences, and generation duration, with compatible context budgets matched. After initialization, histories use generated frames without future ground truth. nKPE at each time averages the preceding one-second window. Timing matches hardware, precision, and output frame count.

These measurements support action following and outcome prediction over the tested 30-second horizon while making the fidelity loss explicit. Longer horizons and closed-loop rollouts with policy-dependent action changes require separate evaluation.

AuthorsOpenReview ↗

Official Comment by Authors

Question 3: Object motion and contact outcomes against real executions

Thank you for emphasizing direct validation against real robot executions. Paired real-robot evaluations provide evidence for predicting action-induced object motion and visible contact outcomes beyond visual plausibility. In Table R6, Pelican-Sim reduces action-effect error by 33.3% relative to Ctrl-World and 42.9% relative to EnerVerse-AC, while improving contact F1 by 8.5 and 13.0 percentage points, respectively. It also achieves lower average and final object displacement errors.

We collected 200 real-robot executions covering pushing, pick-and-place, and complex contact interactions. These comprise five initial-state groups, each with 40 different action sequences. This grouping enables comparisons of the motion differences induced by different actions from corresponding initial conditions. Pelican-Sim receives the first frame and the corresponding actions without fine-tuning on this evaluation set; future real video is used only as the reference.

Table R6. Object motion, contact, and action effects against real executions.

ModelReference trajectoriesObject ADE ↓Object FDE ↓Contact F1 ↑Outcome BA ↑Action-effect error ↓
Pelican-Sim2002.80%4.10%84.00%86.25%2.40%
Ctrl-World2004.00%5.90%75.50%79.38%3.60%
EnerVerse-AC2004.50%6.70%71.00%75.63%4.20%

Note: All methods use the same reference trajectories. ADE, FDE, and action-effect error are normalized by the image diagonal. Action-effect error measures the discrepancy between predicted and executed differences in object displacement under different actions within an initial-state group. Contact F1 assesses visible contact events; BA is balanced accuracy for predicted versus executed task outcomes.

The action-effect comparison in Table R6 tests whether changes in the action sequence produce the correct changes in predicted object motion. Together with contact and outcome agreement, it supports modeling action-conditioned, observable state transitions. These measurements assess projected motion and visible interaction outcomes, rather than contact forces or complete three-dimensional physical states.

Reviewer_Un8QOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Pelican-Sim 1.0, a unified action-conditioned world model simulator for heterogeneous robots. The core idea is to let two complementary action conditions interact throughout generation via injection into alternating blocks. A fine-tuned VLM evaluator converts predicted outcomes into success estimates, utilities, and rewards, supporting policy evaluation, action selection, and policy optimization. Experiments report strong video-quality gains on AgiBotWorld Beta, RoboMIND, and RoboTwin, and downstream improvements on RoboTwin, including data augmentation from 70.0% to 93.0% and imagined policy optimization from 62.7% to 75.4%.

Critical strengths

S1. Combining two complementary action conditions is a useful design. Numerical joint configurations keep precise robot state but leave visual consequences implicit; camera-aligned skeletons show motion geometry but lose depth and joint detail. Feeding both into the model is a reasonable way to solve this issue.

S2. Video-quality gains are comprehensive and fairly compared. The model leads on all five metrics across all three datasets, with PSNR up by 4.636/2.080/10.343 dB and RoboTwin FVD reduced from 26.600 to 4.980.

Critical weaknesses

W1. The paper’s central architectural choice is undermined by its own ablation. The rejected alternative, dual injection, scores slightly higher than the selected interleaving (22.351 vs 22.276 and 23.901 vs 23.850 PSNR), so interleaving provides no quality gain. I hope some clarification about this.

W2. The sparse-MoE gain has a trivial alternative explanation. Switching from dense to sparse MoE in Table 3 (+1.214 PSNR) changes two things at once: it introduces routing and it enlarges the model. Without a dense baseline, the improvement may simply reflect a larger model.

Questions for authors

Q1. Could the authors clarify why interleaving was selected as the central architectural choice when the ablation shows that the rejected alternative, dual injection, achieves slightly higher PSNR?

Q2. Could authors provide a dense baseline with a comparable parameter count, so as to isolate the effect of routing from the effect of model enlargement?

3 comments
AuthorsOpenReview ↗

Official Comment by Authors

Question 1: Interleaving versus dual injection at every block

Thank you for highlighting the higher PSNR of dual injection. We clarify the choice of our default schedule and provide parameter and latency measurements to quantify the quality–cost trade-off.

Dual injection gives the highest PSNR, while interleaving reduces parameter count and denoising latency. We select interleaving as the default to balance prediction quality and inference cost. As Table R1 shows, dual injection improves PSNR by 0.075 dB on AgiBotWorld Beta and 0.051 dB on RoboMIND. Relative to dual injection, interleaving uses 3.37B fewer parameters (23.70%) and reduces denoising time by 11.65 s per clip (20.03%).

To distinguish the contribution from its scheduling choice, we will describe the core design more precisely as hierarchical interaction between joint-level numerical configurations and whole-arm visual geometry. Numerical conditions apply scale-shift modulation to shared hidden states, while visual conditions contribute geometry through residual updates. Interleaving applies the two pathways in alternating blocks; dual injection activates both in every block. Both schedules implement this conditioning framework.

We measured generation-network parameters and denoising latency with matched hardware, precision, video length, resolution, sampling steps, and MoE configuration.

Table R1. Prediction quality and cost of the two injection schedules.

Injection scheduleAgiBotWorld Beta PSNR ↑ (dB)RoboMIND PSNR ↑ (dB)Generation-network parameters (B)Median denoising latency (s/clip) ↓
Interleaved22.27623.85010.8546.51
Dual at every block22.35123.90114.2258.16

Note: PSNR values are from original Table 2. Parameter counts cover the generation network. Time is median denoising latency, rather than end-to-end video generation time.

Table R1 quantifies the quality-cost trade-off; the small PSNR gaps alone do not establish statistical equivalence. Interleaving also outperforms token concatenation and late fusion under the same action conditions in the original ablation, supporting the conditioning design. We retain interleaving as the default and dual injection as the higher-PSNR option among the configurations tested.

AuthorsOpenReview ↗

Official Comment by Authors

Question 2: Separating capacity, computation, and routing

Thank you for asking us to distinguish model capacity from routing.

We added a parameter-matched dense baseline, Dense-P, and a token-independent routing control. These experiments show that a larger total parameter count alone does not explain the MoE advantage, and input-dependent expert selection provides an additional quality gain. The original dense baseline trained on real and simulated data approximately matches MoE in forward computation; we denote it as Dense-F.

Capacity matching. In Table R2, enlarging Dense-F to Dense-P improves PSNR by 0.395 dB and reduces FVD by 2.03, while increasing computation. However, with Dense-P and learned-routing MoE matched in parameters to within 0.005%, MoE still gains 0.819 dB, reduces FVD by 4.50, and uses 36.77% fewer forward FLOPs. This comparison measures the combined effects of sparse structure and routing at comparable total capacity.

Routing control. The two MoE variants in Table R2 share the expert bank, top-2 activation budget, shared expert, action conditions, and training protocol. Their parameter counts are identical and FLOPs differ by less than 0.005%. Learned routing improves PSNR by 0.323 dB and reduces FVD by 2.01, more directly supporting the contribution of input-dependent expert selection.

Forward-compute matching. Learned-routing MoE and Dense-F differ in FLOPs by approximately 0.0047%. MoE improves PSNR by 1.214 dB and reduces FVD by 6.53 at this nearly equal budget. This reflects capacity allocation, expert structure, and routing jointly; actual inference efficiency requires latency measurements.

Table R2. Capacity, computation, and routing controls on AgiBotWorld Beta.

ModelParameters (B)GFLOPs/forwardPSNR ↑ (dB)FVD ↓
Dense-P10.8551882,905.9321.45715.35
Dense-F (original dense baseline)5.2179558,206.4021.06217.38
MoE: token-independent routing10.8555558,206.4021.95312.86
MoE: learned routing10.8555558,232.8322.27610.85

Note: Dense-P/F replace the RGB backbone's MoE FFNs with dense FFNs of intermediate width 73,728/24,576. Other components, action inputs, data, training budget, and generation settings are fixed. Parameters include all experts. GFLOPs count major matrix and attention operations in one DiT forward for 93 frames at 480×640480 \times 640, batch size 1, using two FLOPs per MAC; text encoding, VAE operations, and sampling overhead are excluded.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer Un8Q,

We sincerely appreciate your careful assessment and your recognition of the complementary action representations and the breadth of our prediction results. Your questions prompted us to clarify the distinction between the conditioning framework and its default schedule, and to add comparisons that separate total capacity, computation, and routing. These additions make both the architectural contribution and its measured trade-offs more precise.

Thank you again for your time and constructive suggestions. We hope the additional evidence addresses your concerns and provides a clearer basis for evaluating Pelican-Sim.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.