acceptodds
Withdrawn from ICLR 2027

OmniBrain-0: A Unified Agentic Model for Embodied Intelligence

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

General-purpose embodied intelligence requires not only individual skills but also long-horizon planning, skill coordination, and recovery toward task goals. Specialist robot policies alone do not ensure task-level reliability, while powerful multimodal agents depend on costly long-context reasoning and external experience that compact models cannot readily reproduce. We introduce **OmniBrain-0**, a lightweight unified embodied agent that brings manipulation and navigation into a shared decision framework. Rather than replacing specialist controllers, OmniBrain-0 orchestrates heterogeneous skills through a single high-level model, integrating task understanding, planning, progress tracking, and feedback-driven recovery. To transfer and integrate these capabilities, we develop a progressive training framework combining embodied pretraining, domain-specialized agent distillation, and multi-expert consolidation with MOPD. This framework first converts powerful teachers' interaction experience into specialized decision capabilities, then consolidates manipulation and navigation expertise into a compact unified agent. Crucially, unification occurs within the learned decision model rather than only at the tool interface. We evaluate OmniBrain-0 on LIBERO-Pro, RoboCasa365, RoboTwin 2.0, and R2R-CE Val-Unseen, covering manipulation and navigation. We further evaluate OmniBrain-0 on three real-world manipulation tasks, demonstrating its applicability to physical robot execution. Results show strong performance across both domains and substantial gains over the compact zero-shot backbone, supporting effective transfer and consolidation of agentic planning and recovery capabilities across task domains.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Unified Agentic Robot, Long-Horizon Planning, Multi-Teacher On-Policy Distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_zprhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work introduces OmniBrain-0, a framework designed to solve embodied manipulation, navigation, and combined tasks more efficiently. It incorporates a specialized training method to distill a VLM into a more deployable model. The system is evaluated on benchmarks for manipulation, navigation, and combined tasks, demonstrating its performance across all these benchmarks.

Critical strengths
  1. The work develops a relatively comprehensive execution system and evaluates it across a range of benchmarks.
  2. The experimental results demonstrate performance improvements over several end-to-end baselines on the evaluated benchmarks.
  3. The work designs a VLM training pipeline and conducts corresponding ablation studies to demonstrate the effectiveness of the training approach.
Critical weaknesses
  1. Videos of the real-world robot experiments and more specific quantitative metrics are missing.
  2. The performance gains over conventional methods appear to come primarily from the agentic framework, given the substantially higher performance achieved when using GPT 5.6 sol. Should the contribution of this framework to the performance gains also be analyzed?
  3. GPT-6-Astra has demonstrated substantially stronger overall capabilities than other models on embodied intelligence tasks. Comparisons and analyses involving GPT-6-Astra should be included to establish a more reliable performance upper bound.
  4. The paper’s main technical contribution appears to lie in distillation from closed-source models. Should the experiments therefore focus more on comparing different distillation approaches, including simpler and more direct methods?
  5. Some minor comments: All figures should use vector graphics so that their contents can be selected. Both the text and the figures should clarify which existing approaches are most relevant for comparison with the proposed method: different agentic architectures or different distillation methods. The performance gains and the effectiveness of distillation should be examined more thoroughly through these comparisons, rather than only through comparisons with existing end-to-end approaches.
Questions for authors
  1. Provide more detailed information and results from the real-world robot experiments.
  2. Conduct more fine-grained ablation studies of the agentic design to identify the specific reasons for the substantial gains from introducing GPT 5.6 sol. Further explain how distillation enables a smaller open-source model to approach the performance gains achieved with the closed-source model.
  3. Include corresponding experiments with GPT-6-Astra.
  4. Compare different distillation methods to demonstrate the effectiveness of the proposed approach, rather than only comparing against a setting without distillation.
1 comment

Q1&W1. Real-World Robot Experiments.

We thank the reviewer for the suggestion and provide additional implementation and quantitative details. Our real-world system uses a Piper-X arm with ROS 2, OpenPI's π0.5 low-level VLA fine-tuned on 2,692 demonstrations spanning 113 pick-and-place tasks. π0.5 and OmniBrain-0 run as separate inference servers on NVIDIA RTX 4090 and H20 GPUs, respectively. We evaluate block stacking, apple placement, and drawer opening, with 20 independent trials per task (60 per method), with success determined by completion within the allotted execution budget.

Success rates for OmniBrain-0 are 75%/60%/85%, versus π0.5 at 50%/40%/85% and GPT-5.6-Sol at 90%/75%/95%, respectively. Mean execution time across real-world tasks is:

MethodAvg. Execution Time (min)
π0.50.5
OmniBrain-03.2
GPT-5.6-Sol14.5

OmniBrain-0 improves upon π0.5 by 25 and 20 percentage points on block stacking and apple placement, respectively, while matching its success on drawer opening. Although GPT-5.6-Sol achieves higher success rates, OmniBrain-0 executes tasks approximately 4.5× faster, demonstrating a favorable performance–efficiency trade-off in real-world manipulation. We will add setup details, timings, and demonstration videos to the supplement.

Q2&W2. Agentic Framework Contributions and Knowledge Distillation.

We clarify the gains of the agentic framework from three complementary perspectives. First, comparison with end-to-end VLA baselines captures the contribution of the agentic execution paradigm: high-level reasoning coordinates low-level VLA skills via structured tool calls, supporting more flexible execution of complex embodied tasks. Second, comparisons between GPT-5.6-Sol and GLM demonstrate that stronger foundation models can further improve performance under the same agentic framework. Third, we quantify the contribution of experience augmentation through an Experience Library ablation:

MethodLIBERO-ProRoboTwin 2.0RoboCasa365
w/o Experience Library50.684.426.0
w/ Experience Library78.391.252.4

Success rates (%). Access to the Experience Library raises success by 27.7, 6.8, and 26.4 percentage points, respectively. Together, these comparisons attribute gains to the agentic framework, backbone capabilities, and experience augmentation.

Regarding distillation, we do not aim to reproduce the teacher's general reasoning or broad world knowledge. Instead, we transfer task-specific decisions demonstrated in embodied interactions: task decomposition, skill selection, tool invocation, observation-based reasoning, and feedback-driven correction. This focus helps our smaller student learn embodied capabilities. Rather than training on raw teacher interactions, our pipeline uses Normalization, Quality Control, Reasoning Reconstruction, and Consistency Validation. These steps turn trajectories into consistent supervision, reducing noise and ambiguity in reasoning and tool-use patterns. Progressive Training and MOPD consolidate this knowledge across manipulation and navigation within a unified 9B decision model.

Q3&W3. Regarding GPT-6-Astra Evaluation.

To further evaluate the performance potential and generalizability of our framework, we conducted additional experiments with GPT-6-Astra across all four benchmarks. The results are summarized below.

MethodLIBERO-ProRoboTwin 2.0RoboCasa365R2R-CE Val-Unseen
GPT-6-Astra85.993.058.675.0

Overall success rates (%).

Compared with GPT-5.6-Sol, GPT-6-Astra improves performance by 7.6, 1.8, 6.2, and 4.0 percentage points on LIBERO-Pro, RoboTwin 2.0, RoboCasa365, and R2R-CE, respectively. Our primary goal is to distill proprietary models' embodied capabilities into compact open-source models, narrowing their performance gap. Our framework can accommodate stronger teachers such as GPT-6-Astra, potentially further improving the capabilities of the distilled student model. We will provide corresponding distillation results in future work.

Q4&W4. Effectiveness of Progressive Training and MOPD.

We added a Joint SFT baseline that directly combines manipulation and navigation expert trajectories to train a single model, without our progressive training and MOPD consolidation:

MethodLIBERO-ProRoboTwin 2.0RoboCasa365R2R-CE Val-Unseen
Joint SFT59.584.032.052.0
OmniBrain-0 (MOPD)65.587.239.665.0
Improvement+6.0+3.2+7.6+13.0

Success rates (%); these consistent gains in both domains indicate that improvements cannot be attributed merely to using a stronger teacher. The progressive strategy first develops domain-specialized decision capabilities, after which MOPD consolidates them into a shared policy; this produces more effective cross-domain integration than straightforward joint supervised fine-tuning.

Reviewer_JjqAOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

OmniBrain-0 is a Qwen3.5-9B agent that calls VLA, VLN, perception and motion tools in the Harness VLA framework for manipulation and navigation. It uses embodied QA pretraining, domain-specific SFT on filtered GPT-5.6-Sol / GLM-5.3-Flash trajectories, and MOPD to consolidate two teachers. It improves over zero-shot Qwen3.5-9B on four benchmarks, roughly matches GLM-5.3-Flash, and reports shorter LIBERO-Pro execution time (1.6 vs. 8.5 min).

Critical strengths
  1. A practical goal with a promising efficiency gain: similar overall LIBERO-Pro success to GLM-5.3-Flash (65.5 vs. 64.9) at 1.6 min per task instead of 8.5 (13.8 for GPT-5.6-Sol).
  2. Large gains over the same backbone (LIBERO-Pro 15.6 → 65.5, R2R-CE SR 34.0 → 65.0), using approximately 3.8K agentic samples plus 500K embodied QA samples.
Critical weaknesses
  1. The shared model demonstrates multi-domain capability, but the benefit of consolidation remains unclear without Stage-II teacher and merged-data multi-task SFT baselines. Cross-domain transfer and combined manipulation–navigation tasks are also not evaluated. The appendix provides data-processing, pretraining and VLA-skill ablations. In particular, the recovery-data ablation (56.5 vs. 65.5 on LIBERO-Pro) supports the value of recovery supervision, but does not establish how well specialist capabilities are retained after consolidation.

  2. Appendix E specifies most evaluation counts and states that R2R-CE Val-Unseen environments are excluded from training. However, the separation of agentic training trajectories and evaluation episodes, and the Experience Library's construction and test-time use, remain insufficiently specified, particularly for LIBERO-Pro and RoboTwin. RoboCasa's held-out templates are described relative to pretraining; their exclusion from agentic SFT is unclear. Uncertainty estimates are still missing despite small margins: 65.5 vs. 64.9 on LIBERO-Pro, 87.2 vs. 87.0 on RoboTwin, and R2R-CE SR 65.0 vs. 64.3 for DualVLN, with lower SPL.

  3. Task runtime does not separate decision latency, tool execution and robot waiting time; control frequency is unspecified. The paper should include experiments with both GPT-6 Astra as the planner and the π0.5–Astra selective-assistance framework, using matched tasks, low-level policies, observations and execution budgets.

Questions for authors
  1. Please specify training/evaluation separation for agentic trajectories, especially on LIBERO-Pro and RoboTwin. Are RoboCasa Composite-Unseen templates excluded from agentic SFT? Is the Experience Library fixed before evaluation, and what data does it contain?
  2. Can you report Stage-II teachers and merged-data multi-task SFT on all four benchmarks under comparable training budgets?
  3. Table 8 identifies LingBot-VLA as the full model's RoboTwin tool. Which low-level policies are used elsewhere? Do LLM baselines receive the same observations, tools, prompts, feedback and execution budgets?
  4. Can you provide uncertainty estimates and specify the number of R2R-CE evaluation episodes?
  5. What are the decision latency, low-level action-update and control frequencies, and inference hardware settings? Does execution continue during planning?
  6. Please add matched experiments with GPT-6 Astra and π0.5 with selective Astra assistance, reporting success, decision latency, end-to-end runtime and inference cost per successful task.
1 comment

Q1&W2. Training–Evaluation Separation and Experience Library.

Training and evaluation use disjoint execution instances:

  • LIBERO-Pro: Expert trajectories are collected from standard LIBERO tasks and corresponding Task/Swap variants using seeds s11–s40; evaluation uses s1–s10 across the same eight T/S variants.
  • RoboTwin 2.0: Each task uses five training and five evaluation seeds per Clean/Randomized setting, strictly disjoint and verified as solvable by RoboTwin's built-in scripted expert.
  • RoboCasa365: Training uses Atomic-Seen s11–s40 and Composite-Seen s6–s20; evaluation uses Atomic-Seen s1–s10 and Composite-Seen/Unseen s1–s5. Composite-Unseen templates are excluded from training.
  • R2R-CE: Expert trajectories use only the official Train split; evaluation uses 100 randomly sampled Val-Unseen episodes.

The Experience Library is a text-only repository of execution principles, task strategies, failure recovery, and tool capabilities/limits. It is refined during expert data collection and frozen before evaluation. All LLM baselines use this fixed library on held-out episodes, without updates. It helps collect successful trajectories for distillation, but OmniBrain-0 does not access it at test time: its capabilities are internalized in model weights.

Q2&W1. Effectiveness of Expert Consolidation and Comparison with Joint SFT.

We added comparisons against Stage-II domain experts and Joint SFT, which directly mixes manipulation and navigation expert trajectories:

MethodLIBERO-ProRoboTwin 2.0RoboCasa365R2R-CE Val-Unseen
Stage-II Expert71.888.843.868.0
Joint SFT59.584.032.052.0
OmniBrain-0 (MOPD)65.587.239.665.0

Success rates (%). MOPD exceeds Joint SFT by 6.0/3.2/7.6/13.0 percentage points while remaining only 6.3/1.6/4.2/3.0 points below the respective Stage-II experts.

Q3. Low-Level Policies and Evaluation Fairness.

The fixed low-level policies are π0.5 (LIBERO-Pro), LingBot-VLA (RoboTwin 2.0), RLDX-1 (RoboCasa365), and SmartWay (R2R-CE). GPT-5.6-Sol, GLM-5.3-Flash, and zero-shot Qwen baselines share identical observations, tools, prompts, feedback, budgets, and access to the frozen Experience Library. OmniBrain-0 uses the same setup except that it has no library access or file-reading tool.

Q4&W2. Statistical Uncertainty and Evaluation Episodes.

We ran each benchmark five times and report success-rate mean ± standard deviation:

BenchmarkEpisodes/runMean ± Std (%)
LIBERO-Pro80065.05 ± 0.87
RoboTwin 2.050086.48 ± 1.10
RoboCasa36534038.64 ± 1.15
R2R-CE Val-Unseen10062.60 ± 5.59

R2R-CE uses a fixed subset of 100 episodes randomly sampled from Val-Unseen for all evaluation runs.

Q5&W3. Decision Latency, Control Frequency, and Inference Configuration.

Taking LIBERO-Pro as a representative example, both agents use the same π0.5 low-level policy:

MetricOmniBrain-0GPT-5.6-Sol
Mean Decision Latency (s)2.3831.0
Actions per Chunk55
Chunk Execution Time (s)0.310.31
End-to-End Runtime (min)1.613.8

Times are wall-clock. The simulator runs at 20 Hz (simulation time), with five actions per chunk yielding a nominal 4-Hz chunk-update rate. OmniBrain-0, VLA, and simulator each run on a separate NVIDIA H20 GPU; GPT-5.6-Sol uses an API, with VLA/simulator hardware unchanged.

Planning and execution are synchronous: the agent waits for tool completion, and chunks finish before the next decision; no overlap occurs.

Q6&W3. Comparison with GPT-6 Astra and Selective Assistance.

We thank the reviewer for this valuable suggestion. We conducted additional matched experiments using GPT-6 Astra as the high-level planner on LIBERO-Pro:

MetricGPT-6 Astra
Success Rate (%)85.9
Decision Latency (s)25.3
End-to-End Runtime (min)10.5
Inference Tokens2.3M

End-to-end runtime and inference token consumption are averaged over successful tasks.

These results provide an additional reference for evaluating the performance and inference overhead of a stronger high-level planner.

The cited π0.5 + selective Astra method additionally requires competence labels from policy continuations, training/calibration of task-specific CALL detectors, and a control-handoff/recovery mechanism. Faithful matched reproduction thus entails additional data generation, training, and system integration beyond planner substitution, which we could not complete within the rebuttal period given compute/API constraints. We acknowledge this limitation and plan to release our extensible framework for future comparisons.

Reviewer_DP2dOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents OmniBrain-0, a compact 9B parameters agent model that orchestrates existing manipulation and navigation skills through a shared decision model.

The proposed method involves three training stages: embodied pretraining, domain-specific trajectory distillation, and multi-teacher on-policy distillation (MOPD) to consolidate manipulation and navigation expertise into a lighter-weight model.

The evaluation demonstrates significant improvement over the zero-shot Qwen3.5-9B backbone across three manipulation benchmarks (LIBERO-Pro, RoboCasa365, RoboTwin 2.0) and a navigation benchmark (R2R-CE) and also includes three real-world manipulation tasks. However, performance remains consistently below zero-shot GPT-5.6-Sol.

Critical strengths

The problem of designing an unified model that support high-level reasoning and crosses manipulation and navigation is very important, especially targeting smaller models potentially directly deployed on target hardware.

Using expensive/slow teacher model or agents to produce closed-loop interaction experience, followed by specialization and consolidation, is a reasonable direction toward such deployable embodied systems.

The experimental results shown over the four benchmarks are promising when compared to the utilized backbone in zero-shot.

The proposed approach pragmatically preserves specialist low-level skills rather than trying to produce a single policy to learn all aspects of manipulation and navigation along with task reasoning.

Critical weaknesses

W1. The paper does not convincingly demonstrate "unified embodied intelligence"

This is my principal concern. The presented system integrates manipulation and navigation as tools within a single high-level decision model, but the experiments evaluate them largely as separate domains. There is no convincing demonstration that the system can combine both capabilities within a single genuinely integrated or long-horizon task.

Training one model on examples from two domains and evaluating its performance on those domains establishes multi-domain competence, but not its compositional unification. The distinction is important because the paper repeatedly emphasizes that unification occurs within the learned model rather than merely the tool interface. For proper evaluation, new tasks should be introduced requiring navigation, manipulation, navigation again, and recovery from failures across skill boundaries.

W2. The role of MOPD and the progressive training strategy is not adequately isolated

The overall method combines embodied pretraining, expert interaction data, specialized SFT, and MOPD. However, the main experimental results do not establish how much each component contributes. Especially MOPD, which is an existing method, making this question particularly important to the novelty claims in the paper. Even checking the supplemental material does not further examine this.

W3. The evidence for learned recovery capabilities and generalization is weak

The authors state that successful recovery trajectories are incorporated during distillation, and Figure 4 provides an illustrative recovery example. However, the experiments do not properly show whether the student learned a transferable recovery mechanism or merely better task-conditioned skill selection. A successful final outcome does not necessarily establish effective progress monitoring or recovery.

Also, Section 4.1 reports that teacher trajectories were collected from the same four benchmarks used for evaluation. This is not necessarily problematic, but the text needs to be clear about which tasks, scenes, and perturbations were seen during training and which were held out. Any generalization claims are harder to evaluate without this information.

Questions for authors
  • What empirical evidence demonstrates that MOPD is better than straightforward joint fine-tuning of the two domain-specialized datasets or than other approaches to compose use of the skills/tools?

  • Could OmniBrain-0 complete a single task requiring interleaved manipulation and navigation, including recovery from cross-domain failures? This is key for the target scenario claimed in the paper.

  • What fraction of training trajectories contains meaningful recovery, and what happens when these examples are excluded?

  • How are the trajectory-training samples separated from benchmark test configurations?

  • The execution time on LIBERO-Pro is attractive, but efficiency needs a more systematic analysis (e.g., model inference, number of decisions, total task duration). Why introduce this dimension without further in-depth discussion (also on the other benchmarks)? I feel it dilutes the core results from the experiments with no benefit.

2 comments

Q1&W2. Effectiveness of MOPD over Joint Fine-Tuning.

We added a Joint Fine-Tuning baseline that directly mixes manipulation and navigation trajectories to train one model, without progressive specialization or MOPD.

MethodLIBERO-ProRoboTwin 2.0RoboCasa365R2R-CE Val-Unseen
Joint Fine-Tuning59.584.032.052.0
OmniBrain-0 (MOPD)65.587.239.665.0
Improvement+6.0+3.2+7.6+13.0

Success rates (%). Progressively developing specialized capabilities and consolidating them through MOPD integrates heterogeneous knowledge within a shared decision model more effectively than directly mixing datasets.

Q2&W1. Clarification of Unified Embodied Capabilities.

We thank the reviewer for highlighting this important distinction. The primary goal of OmniBrain-0 is to integrate manipulation and navigation capabilities within a shared high-level decision model, enabling unified tool selection and coordination across different embodied tasks. Our evaluations across both domains are designed to demonstrate this multi-domain competence and the effectiveness of capability integration. We agree that tasks involving interleaved navigation and manipulation, particularly cross-domain failure recovery, would provide further evidence of compositional generalization. While such targeted evaluation is beyond the current experimental scope, we appreciate this valuable suggestion and will consider incorporating these more challenging scenarios in future work.

Q3&W3. Effectiveness of Recovery Trajectories.

Recovery trajectories are successful episodes containing a failure followed by feedback-driven correction (e.g., re-localization, grasp adjustment, re-grasping). We measured their prevalence and ablated them from training:

BenchmarkRecovery / TotalRatio (%)w/o RecoveryFull Model
LIBERO-Pro557 / 1,90229.356.565.5 (+9.0)
RoboTwin 2.036 / 4278.486.488.2 (+1.8)
RoboCasa365108 / 61417.631.439.6 (+8.2)

Success rates (%); gains are percentage points. Removing recovery experiences decreases success on every benchmark. Rather than merely providing canonical successful demonstrations, they retain the intermediate failure evidence and subsequent corrective decisions. This teaches the model to adapt tool selection and execution strategies when outcomes deviate from expectations, supporting robustness to recoverable failures.

Q4. Separation of Training and Evaluation Configurations.

Training and evaluation use disjoint execution instances:

  • LIBERO-Pro: We collect expert trajectories from standard LIBERO tasks and their corresponding Task (T) and Swap (S) variants in LIBERO-Pro using seeds s11–s40. Evaluation covers the same eight Task and Swap variants using seeds s1–s10.
  • RoboTwin 2.0: For each task, evaluation is conducted using five seeds under each of the Clean and Randomized settings. Expert trajectories are collected using five additional seeds per setting, strictly disjoint from the evaluation seeds. Both training and evaluation seeds are verified as solvable using RoboTwin's built-in scripted expert.
  • RoboCasa365: Training uses Atomic-Seen s11–s40 and Composite-Seen s6–s20; evaluation uses Atomic-Seen s1–s10 and Composite-Seen/Unseen s1–s5. Composite-Unseen templates are excluded from training.
  • R2R-CE: Expert trajectories are collected exclusively from the official R2R-CE Train split, while evaluation is conducted on 100 randomly sampled episodes from the Val-Unseen split.

Therefore, training and evaluation are separated through disjoint seed configurations for manipulation benchmarks and disjoint dataset splits for R2R-CE.

Q5. Efficiency and Performance Trade-off.

We clarify that the reported execution time measures total task duration, including model decision-making and tool execution. To provide a more systematic comparison, we summarize both task success rates and execution times across all four benchmarks.

BenchmarkMethodSR (%)Time (min)
LIBERO-Proπ0.543.80.6
GPT-5.6-Sol78.313.8
OmniBrain-0 (9B)65.51.6
RoboTwin 2.0LingBot-VLA84.82.8
GPT-5.6-Sol91.210.5
OmniBrain-0 (9B)87.23.7
RoboCasa365RLDX-136.05.3
GPT-5.6-Sol52.414.8
OmniBrain-0 (9B)39.613.3
R2R-CEDualVLN64.30.73
GPT-5.6-Sol71.06.03
OmniBrain-0 (9B)65.00.86

OmniBrain-0 completes tasks 8.6×, 2.8×, 1.1×, and 7.0× faster than GPT-5.6-Sol on these benchmarks, respectively. Compared with specialized VLA/VLN policies, it achieves consistently higher success but takes longer. These results place OmniBrain-0 between fast specialized policies and more capable but slower generalist agents, providing a favorable balance between success rate and total execution time.

I appreciate the authors' for their targeted response and additional experiments. Especially the results showing the impact of no recovery traces, which should be incorporated into the paper text. Thanks also for the comparison to Joint Fine-Tuning. While limited, that's understandable in the limited rebuttal time, and it does provide some supporting evidence for the claims of MOPD benefits.

But my main concern regarding "unified embodied intelligence" and generalization remains. I also checked the response to Reviewer XxkW, who raised similar concerns. I understand the authors point about their definition of "unification" and that needs to be stated clearly in the paper text (and properly differentiated to the more general definition), but I'm not convinced the results provide strong evidence to support the claim following their own definition. The results you show there for LIBERO-Plus Zero-Shot are interesting to illustrate some level of domain generalization, but those tasks are basically variations of what your model has already seen in training data for LIBERO-Pro, not fully novel instances, right?

The results added in the answer to Q5 do not exactly address the issue and are not really a fair comparison as latency for the closed model API cases is highly dependent on deployment environment and networking characteristics. My main point here was that total execution time is expected to be much shorter if you train your own small-ish model, but for real inference time perf. claims, more detailed experiments would be needed. So I wouldn't emphasize this aspect so much as there are no insightful or surprising results here. BTW, these got me curious, why is time in RoboCasa365 so much worse than others (i.e., almost GPT-level latency)?

Also, in the recovery cases discussion, why no cases apply to the navigation benchmark?

The rebuttal addresses some of my concerns, but not enough to justify improving my previous score. But I wouldn't argue strongly against acceptance.

Reviewer_XxkWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents OmniBrain-0, a unified embodied agent for robotic manipulation and navigation. It employs a shared decision-making framework to support high-level decision-making and skill coordination across tasks. To this end, the authors develop a progressive training framework for capability transfer and integration. The framework sequentially combines embodied pretraining, distillation of domain-specific agents, and multi-teacher on-policy distillation to integrate expert capabilities in planning, skill scheduling, and failure recovery from both manipulation and navigation domains into a unified decision model, while preserving the specialized capabilities of the low-level execution policies. The authors evaluate the model on multiple manipulation and navigation benchmark datasets and further conduct real-robot manipulation experiments, demonstrating the feasibility of transferring the model from simulated environments to real-world scenarios.

Critical strengths
  • The paper reframes the integration of cross-domain embodied skills as a problem of unified modeling of high-level decision-making capabilities, offering an insightful perspective. Through a shared decision-making framework, the model can coordinate specialized skills from different domains and learn high-level capabilities such as task decomposition, skill scheduling, execution-state tracking, and failure recovery. This research direction has clear practical significance and provides a feasible solution for the unified coordination of heterogeneous embodied skills.
  • The training pipeline incorporates successful recovery trajectories and structured decision supervision, rather than relying solely on standard successful demonstrations. The ablation results show that both components make substantial contributions to task performance, particularly in scenarios that require feedback-driven correction.
Critical weaknesses
  • The paper’s core methodological contribution is a progressive training framework consisting of embodied pretraining, domain-specific agent distillation, and MOPD-based multi-teacher capability integration. However, the current ablation studies focus primarily on the effects of the data-processing pipeline, the experience repository, embodied pretraining, and the underlying VLA model, and therefore do not sufficiently validate the necessity and effectiveness of the proposed training strategy. In particular, the paper lacks a direct comparison with simpler training strategies. For example, expert trajectories from the manipulation and navigation domains could be mixed and used to train a unified model directly through joint supervised fine-tuning.
  • Although the paper proposes an embodied agent that unifies manipulation and navigation capabilities, the current experiments still evaluate the two domains separately. The paper does not demonstrate whether the model can coordinate manipulation and navigation skills within the same long-horizon task. The existing results show that the shared high-level decision-making model can handle manipulation and navigation tasks independently. However, they are insufficient to establish that the model possesses cross-domain skill-coordination capabilities, and they do not clearly demonstrate the additional benefits brought by integrating capabilities from multiple domains.
Questions for authors
  • Have the authors attempted to directly combine expert trajectories from the manipulation and navigation domains and train a unified model using joint SFT? Under the same training data and a comparable computational budget, how does this strategy perform relative to the proposed progressive training framework? In addition, are there results from an experiment that removes MOPD alone, in order to isolate and verify the specific contribution of multi-teacher on-policy distillation to the final performance?
  • The paper proposes an embodied agent that unifies manipulation and navigation decision-making capabilities. However, the real-robot experiments reported so far only involve manipulation tasks, while the evaluation of navigation capabilities is limited to simulated environments. Have the authors conducted navigation experiments on a real-robot platform?
  • The paper evaluates the model on multiple manipulation and navigation benchmarks, but the training stage also uses expert interaction trajectories collected from these benchmarks. It is therefore unclear whether the model has been evaluated for out-of-distribution generalization across benchmarks. Have the authors conducted such cross-benchmark generalization experiments?
3 comments
AuthorsOpenReview ↗

Author Response

Q1&W1. Effectiveness of Progressive Training and MOPD.

We thank the reviewer for this valuable suggestion. To better evaluate our training strategy, we conducted additional experiments comparing OmniBrain-0 with a Joint SFT baseline, which directly mixes manipulation and navigation expert trajectories to train a unified model without MOPD. We also report the performance of OmniBrain-Base and the Stage-II domain-specialized experts to further examine the contributions of different training stages.

MethodLIBERO-ProRoboTwin 2.0RoboCasa365R2R-CE Val-Unseen
OmniBrain-Base55.378.828.445.0
Stage-II Experts71.888.843.868.0
Joint SFT59.584.032.052.0
OmniBrain-0 (MOPD)65.587.239.665.0

Success rates (%). Stage-II Experts are separately trained domain-specific models, whereas Joint SFT and OmniBrain-0 are unified models.

Comparison with Joint SFT. OmniBrain-0 consistently outperforms direct Joint SFT across all four benchmarks, achieving improvements of +6.0, +3.2, +7.6, and +13.0 percentage points, respectively. These results demonstrate the advantage of progressive specialization and MOPD-based consolidation over directly mixing expert trajectories, across both manipulation and navigation domains.

Contribution of MOPD and capability consolidation. The Stage-II experts achieve strong domain-specific performance (71.8/88.8/43.8/68.0), providing a reference for evaluating capability retention during consolidation. Without MOPD, these capabilities remain distributed across separate specialized models. Through MOPD, OmniBrain-0 integrates them into a single unified 9B model, retaining much of their performance (65.5/87.2/39.6/65.0). Compared with Joint SFT, MOPD uses token-level distributional supervision from specialized teachers on student-generated prefixes, enabling more effective integration of their decision-making capabilities.

Together, these results support the effectiveness of our progressive training strategy in acquiring specialized expertise and consolidating it into a unified embodied agent, while mitigating the performance degradation associated with direct joint training.

Q2&W2. Real-World Navigation Evaluation.

We thank the reviewer for this valuable suggestion. Our current real-world experiments primarily focus on manipulation, while navigation capabilities have been systematically evaluated on R2R-CE Val-Unseen. To further validate the practical applicability of OmniBrain-0, we are currently extending our real-world evaluation to navigation tasks. We plan to provide supplementary demonstration videos documenting the model's navigation behavior and closed-loop decision-making on a physical robot. These additional evaluations will further complement our existing results and provide a more comprehensive assessment of OmniBrain-0 in real-world embodied scenarios.

Q3. Cross-Benchmark OOD Generalization on LIBERO-Plus.

We thank the reviewer for highlighting the importance of cross-benchmark generalization. To further evaluate this capability, we conducted additional zero-shot experiments on LIBERO-Plus, a challenging benchmark comprising task variants across seven perturbation types: camera viewpoints, robot initial states, language instructions, lighting, background textures, sensor noise, and object layouts.

We evaluate four task suites (SPATIAL, OBJECT, GOAL, and LIBERO-10), sampling 210 task variants per suite, evenly distributed across the seven perturbation types (30 tasks per type per suite), for a total of 840 evaluation episodes.

LIBERO-Plus Zero-Shot Evaluation (Success Rate, %)

MethodCameraRobotLanguageLightBackgroundNoiseLayoutOverall
$\pi_{0.5}$69.274.283.394.295.083.385.083.4
Qwen3.5-9B12.515.815.810.815.013.316.714.3
OmniBrain-0 (9B)70.075.885.892.595.882.584.283.8

Without any additional training on LIBERO-Plus, OmniBrain-0 (9B) achieves an overall success rate of 83.8%, substantially outperforming the untrained Qwen3.5-9B backbone (14.3%, +69.5 percentage points) and achieving comparable performance to the specialized $\pi_{0.5}$ baseline (83.4%). Notably, OmniBrain-0 surpasses $\pi_{0.5}$ under four perturbation types: Camera (+0.8), Robot (+1.6), Language (+2.5), and Background (+0.8 percentage points).

These results provide additional evidence that the learned high-level decision-making capabilities transfer to a separately evaluated benchmark without benchmark-specific fine-tuning, while maintaining robust performance across diverse visual, linguistic, and physical perturbations.

I thank the authors for their detailed response and additional experiments. The rebuttal addresses some of my concerns, particularly regarding the effectiveness of the proposed training strategy. While the additional Joint SFT comparison strengthens the empirical justification of the proposed training framework, my main concern regarding cross-domain skill coordination remains unresolved. The current results demonstrate that a unified model can perform manipulation and navigation tasks separately, but do not establish its ability to coordinate these skills within a single long-horizon task. Could the authors provide experimental evidence of such integrated behavior, or clarify the scope of the claimed unification?

We thank the reviewer for the follow-up and acknowledgment of our additional results. We would like to clarify that unification in OmniBrain-0 refers to integrating manipulation and navigation capabilities into a single shared high-level decision model, enabling one compact agent to handle diverse tasks across both domains. Our primary focus is on learning and consolidating complementary capabilities within a unified model, rather than emphasizing the coordination of manipulation and navigation within a single task. We recognize that our current presentation may not have made this distinction sufficiently clear and will refine the wording in the revised manuscript. We also appreciate the reviewer's insightful suggestion, as cross-domain skill coordination is a meaningful research direction that we hope to explore in future work on suitable robotic platforms.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.