acceptodds
Withdrawn from ICLR 2027

One State, One Roll: Unified Physical Latent Flow for Robot Manipulation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Embodied foundation models are increasingly moving toward multimodal prediction, jointly covering actions, proprioceptive states, and visual observations. Yet these modalities are usually generated through separate modality-specific pathways. Such separation makes cross-modal physical consistency an additional coordination problem: different pathways can predict incompatible futures, while each modality is evolved without a shared representation of the physical transition. Moreover, most models generate every future chunk from random noise, leaving the local continuity of an already realized physical trajectory largely unused. We introduce OneRoll, a unified physical latent flow for robot manipulation. OneRoll linearly fuses the three modalities into one physical token and recovers them through linear projections. The token's velocity on the physical manifold is thus the sum of their linearly mapped derivatives, directly supervising a shared vector field whose integration extends the realized trajectory into the future. Experiments on both simulated and real-robot tasks show encouraging performance, suggesting that unified physical evolution is a promising foundation architecture for embodied models.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot manipulation;embodied foundation model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2TYTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes OneRoll, a unified physical latent flow model for robotic manipulation. The method maps actions, proprioceptive states, and visual observations into a shared latent space and combines them through linear addition. A shared vector field and flow matching are then used to predict future latent states, enabling unified modeling of multimodal physical dynamics. The authors further introduce training objectives for velocity supervision, future-modality reconstruction, latent consistency, and cycle reconstruction. Experiments are conducted on LIBERO-X, EBench, RoboLab-120, and real-robot manipulation tasks. The reported results demonstrate competitive task success rates, and comparisons with methods such as A2A and modality ablations provide evidence of the method's potential. However, the independent contribution of the unified latent evolution mechanism and the verifiability of some experimental results remain unclear.

Critical strengths

The paper explores the idea of unifying actions, proprioceptive states, and visual observations into a single evolving latent representation, using a shared dynamics model to predict future states. This provides a potentially interesting modeling direction for robotic world models. The experiments cover multiple simulation benchmarks, transfer from real-world data, and real-robot manipulation. In particular, comparisons with A2A provide some evidence supporting the potential of the proposed approach. These results demonstrate that the paper goes beyond a conceptual proposal and includes substantial experimental investigation.

Critical weaknesses
  1. Limited methodological novelty. The paper aims to fuse actions, proprioceptive states, and visual observations, which is a reasonable direction. However, the fusion operation in Eq. (4) simply adds the features of the three modalities, which offers little novelty at the level of multimodal fusion methods.
  2. Limited mathematical novelty and potentially redundant objectives. Equation (7) is essentially the derivative of Eq. (4), which appears to be a mathematically correct but straightforward derivation with limited necessity. Equations (9) and (11) also have considerable overlap: the former reconstructs future data from the integrated representation, while the latter reconstructs data from the encoded future representation. More importantly, given that Eq. (9) requires the predictions to reconstruct future data and Eq. (10) requires the predicted representation to approach the encoded future representation, it remains unclear whether the cycle reconstruction loss in Eq. (11) provides indispensable supervision.
  3. Significant concerns regarding the experiments. Around line 352, the paper states that the exact task and trial counts for each LIBERO-X level will be added once the evaluation protocol is finalized. Appendix A.3 similarly uses future tense to indicate that task lists, trial counts, post-training settings, and success criteria will be provided later. Does this mean that the experiments have not yet been completed? Moreover, Table 2 reports success rates of 0% for some tasks and 99% or 100% for others. How do the authors explain these results? In particular, line 374 states that the baseline methods and the latest OneRoll run use 20 trials per task. If so, success rates should change in increments of five percentage points (e.g., 90%, 95%, or 100%), making a reported rate of 99% impossible. This raises concerns about the reliability of the reported results.
  4. Insufficient visual evidence for the real-robot cooking experiments. The real-robot potato-preparation and cooking experiments are interesting and offer evidence beyond simulation, provided that they were conducted as reported. However, the robot images in the appendix appear to lack the relevant objects. For example, the first row depicts pouring oil, and the second depicts adding potato shreds, yet neither the oil bottle nor the potato shreds are visible. These images do not clearly substantiate the claimed manipulation steps. In contrast, Figure 3 at least shows potatoes in a container.
  5. Insufficient isolation of the core contribution. The ablation studies do not establish how much of OneRoll's performance improvement can be attributed specifically to the unified latent evolution mechanism introduced in Eq. (4).
Questions for authors
  • Limited methodological novelty and insufficient validation of the core contribution. Equation (4) primarily performs multimodal fusion by summing linearly projected features, which is a standard operation. Equation (7) follows directly from this linear fusion definition and offers limited theoretical novelty. In addition, the supervision objectives in Eqs. (9)–(11) exhibit some overlap, and their necessity is not sufficiently justified. The existing ablations do not adequately demonstrate that unified latent evolution itself is superior to conventional multimodal fusion followed by separate prediction heads.
  • Important inconsistencies in experimental reporting affect verifiability. Table 2 reports success rates of 0% for some tasks and 99% or 100% for others. How do the authors explain these results? Moreover, line 374 states that the baseline methods and the latest OneRoll run use 20 trials per task. If so, success rates should change in increments of five percentage points (e.g., 90%, 95%, or 100%), making a reported rate of 99% impossible. This raises concerns about the reliability of the reported results. More concerningly, the paper states that each task is evaluated over 20 trials, yet EBench Table 2 reports a success rate of 99.0%. If this metric is calculated directly from 20 binary trial outcomes, such a value is impossible. Around line 352, the paper states that the exact task and trial counts for each LIBERO-X level will be added once the evaluation protocol is finalized. Appendix A.3 similarly uses future tense to indicate that task lists, trial counts, post-training settings, and success criteria will be provided later. Does this mean that the experiments have not yet been completed? The authors need to clarify the actual evaluation protocol and success-rate calculation; otherwise, the reliability of the reported results cannot be adequately assessed.
  • Insufficient isolation of the architectural contribution and limited visual evidence for real-robot results. The paper reports extensive pretraining, but the current experiments do not adequately distinguish the contribution of unified latent evolution from that of the pretraining data and computational scale. Furthermore, although the real-robot cooking experiments are valuable, the appendix images do not clearly show some of the key objects and interactions involved in the claimed manipulation steps. Each subtask is evaluated over only 10 trials, and the overall success rate for completing the entire nine-stage task is not reported. Consequently, the current evidence is insufficient to fully support stronger claims about reliable long-horizon real-world manipulation.
Reviewer_uPGrOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OneRoll, which jointly encodes actions, proprioceptive states, and visual features into a shared latent representation and evolves it with a learned flow. Using realized trajectories to initialize future prediction is intuitive, and the simulation and real-robot results are promising.

Critical strengths
  1. The paper identifies an important limitation of existing robot policies: future predictions are often generated independently instead of continuing from the realized physical trajectory. OneRoll addresses this with a conceptually simple shared latent-flow architecture and linear modality readouts.
  2. The method is evaluated on LIBERO-X, EBench, RoboLab-120, and a real dual-arm manipulation task, covering generalization, dexterous manipulation, real-to-simulation transfer, and physical deployment.
  3. The results suggest that warm-start temporal continuation improves over standard action generation, while state and visual supervision provide complementary gains.
Critical weaknesses
  1. The central claim of physical consistency is insufficiently supported. Additive fusion does not prevent different modalities from occupying disjoint latent subspaces, and no direct consistency metric is reported.
  2. The paper contains several specification and reproducibility issues: LIBERO-X trial counts are unspecified, Appendix A.3 remains a placeholder, Equations (6) and (15) use opposite velocity directions, and the visual feature dimension is inconsistently reported as 512-D and 1,024-D.
  3. Table 2 reports 99% success with 20 trials, which is mathematically inconsistent because success rates should change in increments of 5%.
  4. The real-robot evaluation reports per-subtask results rather than end-to-end long-horizon success.
Questions for authors
  1. Can the authors provide experiments showing that the different modalities learn a genuinely shared latent space rather than occupying independent subspaces?
  2. Can the authors directly evaluate cross-modal consistency—for example, whether the decoded future states and observations agree with the physical consequences of the decoded actions?
  3. How was the 99% success rate obtained from 20 trials?
  4. Were the nine real-robot subtasks evaluated independently from reset states or executed as complete continuous sequences? What is the end-to-end success rate for the complete task?
Reviewer_MP78OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes OneRoll, which puts actions, proprioceptive states, and visual features into a single shared latent. It predicts future chunks by integrating forward from recent interaction history. The model is pretrained on about 30,000 hours of robot data from 11 embodiments. It reaches 45.2% success on LIBERO-X and 51.1% on EBench TableTop. On a real-world potato prep and cooking task, it outperforms the evaluated baselines on each individual subtask.

Critical strengths

The idea of putting all observations, states and actions into one shared latent is interesting. The evaluation is broad, covering several simulation benchmarks plus physical tasks.

Critical weaknesses
  1. The claim of physical consistency is not directly shown. It doesn’t show that executing the predicted actions actually produces the predicted visual changes. It would be more convincing to add a test comparing the difference between the predicted visual state and the actual achieved visual state.
  2. The performance gain is limited. The gain on Libero-X is okay but on EBench TableTop it only outperforms by 1.1%. Without error bars, the small performance gain is not statistically meaningful.
  3. The importance of the shared latent is not studied. Another ablation study on comparing the model with separate output heads for each modality is needed.
Questions for authors
  1. The paper says EBench uses 20 trials per task, but two tasks report 99% success, how is this possible? Also A3 is a placeholder without actual details.
  2. Real experiment results are reported as ten trials per separately instructed subtask, not as success on the full sequence. Does each subtask starts from a reset or from the finish state of the previous subtask? What's the full-sequence results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.