acceptodds
Under review as a conference paper at ICLR 2027

TriBrain: A Three-System Embodied Foundation Model for Scalable Robot Manipulation

Abstract

Vision-language-action (VLA) models have become a leading approach to generalist robot control, yet unifying understanding, prediction, and control within an embodied foundation model remains challenging. In particular, staged VLA training separates vision-language adaptation from continuous-action learning, limiting the co-adaptation of semantic representations and continuous control. Moreover, integrating world models into VLA pipelines for data augmentation or visual subgoal prediction, without coupling future-state prediction with task-progress evaluation, limits their effectiveness in guiding long-horizon manipulation. To address these issues, we present TriBrain, a multi-embodiment VLA pretraining framework. The model adopts a three-system architecture: System 1 converts multimodal context into continuous robot actions; System 2 performs scene understanding and task decomposition; and System 3 predicts future observations and estimates task progress to guide action generation. TriBrain jointly trains a pretrained VLM backbone and a continuous-action expert while incorporating multimodal understanding objectives, mitigating optimization fragmentation. Soft Knowledge Insulation attenuates action gradients entering the vision-language backbone to balance semantic preservation and motor adaptation. During policy post-training, a frozen World Value Model (System 3) provides predicted subgoal images and task-progress estimates used to derive binary advantage labels. The subgoal images and advantage labels serve as policy inputs and are independently dropped during training. Trained on over 37,000 hours of heterogeneous embodied experience, TriBrain surpasses in real-world manipulation, achieving 84.9% and 74.1% success on complex tasks with AgileX PiPER and Maker H01, respectively. In simulation, it outperforms by 9.0% and 5.22% on RoboTwin 2.0 and EBench, respectively. In embodied vision-language evaluation on MiMo-Embodied, it also achieves state-of-the-art performance with an overall score of 0.4621. Code and pretrained weights will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-language-action models, Robot learning, Multi-embodiment learning, World models, Long-horizon manipulation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_q2UqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents TRIBRAIN, a multi-embodiment vision-language-action framework designed to jointly address semantic understanding, continuous action generation, and predictive execution guidance. The model consists of three functional systems: SYSTEM 1 generates continuous robot actions, SYSTEM 2 performs task understanding and subtask decomposition, and SYSTEM 3 predicts future visual subgoals and estimates temporal task progress. To facilitate joint vision-language-action learning, the paper introduces Soft Knowledge Insulation, which attenuates action-loss gradients entering the vision-language backbone. During policy post-training, the frozen World Value Model in SYSTEM 3 provides visual subgoal and progress-aware guidance to the action policy, with independent dropout used to improve robustness. The paper conducts extensive experiments across foundation-model evaluation, real-robot post-training, simulation benchmarks, and embodied spatial/affordance understanding. Results are reported on multiple robot embodiments, including AgileX PiPER and Maker H01, as well as benchmarks such as RoboTwin 2.0, EBench, RoboColiseum, and MiMo-Embodied. The large-scale evaluation, together with ablations on architectural decoupling, Soft KI, visual subgoals, and progress guidance, provides broad empirical evidence for the proposed framework.

Critical strengths
  1. The paper proposes a coherent three-system VLA framework that separates task understanding, action generation, and future-state prediction with progress evaluation. This decomposition addresses important challenges in long-horizon, multi-embodiment robot control.
  2. The evaluation is broad and substantial, covering large-scale pretraining, multiple robot embodiments, real-world manipulation, simulation benchmarks, and embodied spatial and affordance understanding. This breadth makes the empirical claims more consequential than results on a single robot or benchmark.
  3. The paper includes informative ablations of architectural decoupling, Soft Knowledge Insulation, visual subgoals, and progress guidance. These experiments provide evidence that the reported improvements are associated with specific design components rather than only with the overall model size.
Critical weaknesses
  1. TRIBRAIN uses over 37,000 trajectory hours and 272M VL samples, which may be substantially more than the baselines. The paper should clarify the pretraining data and compute budgets, and provide matched comparisons to determine whether the gains arise from the proposed design or simply from larger-scale training.
  2. In SYSTEM 3, the binary progress signal is defined by whether the predicted future value is strictly greater than the current value. For trajectories that are close to completion, small prediction errors could cause the progress labels at adjacent time steps to alternate between 0 and 1. The paper should clarify whether it uses a margin, temporal smoothing, hysteresis, or another stabilization mechanism.
  3. For each benchmark, the paper should specify the pretrained checkpoints, post-training data, and adaptation protocol used by each method, including whether evaluation is zero-shot or fine-tuned.
  4. Since PiPER and H01 are separately post-trained with embodiment-specific masks, the current results do not establish embodiment-general control. Zero-shot cross-embodiment transfer or shared-representation analyses would strengthen this claim.
  5. The experiments are broad and solid, but the paper’s writing does not clearly communicate its main contributions. The roles of the three systems, Soft KI, and dual-signal guidance are not sufficiently distinguished from existing techniques and system-level integration. The paper would benefit from a clearer contribution statement.
Questions for authors
  1. How do the pretraining data and compute budgets of TRIBRAIN compare with those of each baseline? Are the reported gains preserved under data- and compute-matched comparisons?
  2. For each benchmark, which pretrained checkpoint, post-training dataset, and adaptation protocol are used for TRIBRAIN and each baseline? Are the results zero-shot, few-shot, or fully fine-tuned?
  3. In SYSTEM 3, how are per-timestep progress targets constructed, and how is the strict value-difference threshold stabilized against small prediction errors and temporal label flipping?
  4. Since PiPER and H01 are separately post-trained with embodiment-specific action masks, what evidence shows that TRIBRAIN learns embodiment-general control rather than robot-specific policies? Do you have zero-shot cross-embodiment or shared-representation analyses?
2 comments

Response to All Reviewers: We thank reviewers for recognizing our coherent, reasonable three-system framework [EZzK, q2Uq], substantial scale and broad evaluation [EZzK, gFvy, q2Uq], clear, practical real-robot gains [gFvy], and informative component analyses and ablations [gFvy, q2Uq].

W1&Q1: Data- and compute-matched comparisons.

Public information on baseline pretraining data, duration, and compute is inconsistent, so a strict resource-matched comparison of original pretraining may be impossible. We therefore control downstream post-training and conduct capacity-matched experiments to better isolate model-design effects.(1)TriBrain and all baselines in Tables 1–3 use the same downstream datasets and compute, with full-parameter fine-tuning on 8 A100 GPUs. These gains thus do not arise from extra post-training data or compute; they reflect model-design differences under unified post-training conditions.(2)Table 10's capacity-matched experiments further assess model capacity and computational cost. TriBrain retains gains under comparable capacity and cost, further supporting our architectural design.

W3&Q2: Benchmarks and evaluation settings.

TriBrain initializes from our multi-embodiment pretrained checkpoint, while baselines use official pretrained checkpoints and recommended configurations. Specifically: (1)Fig. 4 (real-robot): Evaluates our pretrained TriBrain checkpoint directly on PiPER and H01 without post-training, testing generalization (OOD, zero-shot) and cross-embodiment transfer (ID,few-shot). (2)Tables 1–2 (real-robot, fine-tuned): Uses identical downstream task datasets and full-parameter fine-tuning (8 A100 GPUs, matched compute budgets) (3)Table 3 (simulation, fine-tuned): Fine-tunes all models on each simulation benchmark's official training split under identical observation inputs, camera views, and control frequencies.

Q3: How are per-timestep progress targets constructed.

Per-timestep progress targets are constructed from sparse episodic success/failure labels. For a successful episode ending at terminal step TendT_{\text{end}}, we define the reward as rt=−1r_t = -1 for all pre-terminal steps and rTend=0r_{T_{\text{end}}} = 0 upon completion, whereas failed episodes receive a large negative constant. The empirical return is then discretized and normalized per task into (−1,0)(-1, 0) to train the distributional value function.

W2&Q3: Threshold robustness to prediction errors and label flipping.

Through three complementary mechanisms: (1) Temporal windowing: comparing states separated by Δt\Delta t (aligned with the action chunk) captures macro task advancement rather than reacting to high-frequency, adjacent-frame noise. (2) Continuous value learning: the Value Head regresses continuous expected return targets rather than performing noisy binary classification, providing a smooth underlying representation before discretization. (3) Fixed deployment prior: at inference, we clamp Pt=1\mathcal{P}_t = 1 as a constant positive condition, completely decoupling deployment guidance from online prediction jitter or training-time label flips. Finally, Appendix G empirically confirms that these temporal advantage labels improve policy performance despite discretization.

W4&Q4: Embodiment-general control and cross-embodiment transfer.

Figure 4 demonstrates zero-shot cross-embodiment transfer: the pretrained TriBrain checkpoint directly runs on PiPER and H01 without task-specific adaptation, achieving OOD success rates of 84.4%/60.0% (PiPER) and 30.0%/10.0% (H01). This proves that multi-embodiment pretraining learns transferable manipulation knowledge rather than relying solely on robot-specific post-training. Furthermore, embodiment-specific action masks only handle action dimensions and control degrees of freedom during post-training, meaning visual semantics and task representations are shared rather than relearned from scratch for each robot.

We thank the reviewer for helping improve the final manuscript.

Reviewer_q2UqOpenReview ↗

Response to Author Rebuttal

Thank you for the detailed responses and additional clarifications. The explanations regarding resource-matched comparisons, evaluation settings, progress target construction, and cross-embodiment transfer help address my questions. The additional evidence further strengthens the empirical support for the proposed framework.

Overall, I appreciate the authors' efforts in addressing my concerns and maintain my positive assessment of the paper.

Reviewer_gFvyOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

TriBrain presents a multi-embodiment VLA organized into three systems: System 1 (PaliGemma2-3B with a 0.5B action expert) generates continuous action chunks, System 2 (a separate PaliGemma2-3B) predicts the current subtask, and System 3 (a 5B world value model built on GigaWorld-1) provides predicted subgoal images and a binary progress signal during post-training. Systems 1 and 2 are pretrained jointly in a single stage on about 37K hours of embodied data and 272M vision-language samples, with Soft KI attenuating action-loss gradients into the VLM. After post-training, the model improves over the public π0.5 checkpoint on complex tasks on AgileX PiPER and Maker H01 (89.2 vs. 70.4; 74.1 vs. 45.2) and on RoboTwin 2.0 (67.35 vs. 58.35). Overall, this is a substantial and carefully engineered system paper with clear real-robot gains and useful ablations. The main room for improvement lies in positioning relative to closely related work and in a few additional controls. I lean toward acceptance.

Critical strengths
  1. Substantial scale with clear real-robot gains. The paper evaluates on two real robot platforms, three simulation benchmarks and an embodied vision-language benchmark. The largest real-robot gains appear on multi-stage tasks (on PiPER, Food Prep Heating 50 → 100 and Dish Cleanup 60 → 100), and RoboTwin performance remains strong under domain randomization. Building and evaluating a system at this scale is a significant effort, and the results are practically relevant.

  2. Informative component analyses. The Soft KI sweep (Table 4b) shows that intermediate gradient attenuation preserves vision-language ability while improving control. The shuffled, constant and random progress-label controls (Table 13) support trajectory-aligned progress supervision, and the guidance-corruption tests (Table 12a) show that the policy degrades gracefully when System 3 signals are noisy or missing. Together with the data-scaling and source-composition experiments (Figure 3) and the capacity-matched comparison (Table 10), these analyses offer useful guidance for building multi-embodiment VLAs, even where individual components build on prior work.

Critical weaknesses
  1. Positioning relative to closely related work could be sharpened. TriVLA (arXiv 2507.01424) also adopts a three-system design in which a video diffusion model serves as a world-model/dynamics module; π0.7 conditions its policy on world-model-generated subgoal images and episode metadata with prompt dropout; and RECAP (π0.6∗\pi^*_{0.6}) conditions on a binarized value-derived advantage and uses a positive indicator at inference, similar in spirit to fixing Pt=1P_t = 1. Discussing these works, and slightly revising the descriptions of KI (which stops action-expert gradients rather than freezing the visual backbone, §2) and of π0.7 (§1, §2, §3.2), would help readers see the paper's specific contributions more clearly: attenuated rather than stopped action gradients, progress labels derived from imagined future values, and their integration at scale.

  2. Additional controls and a few clarifications would strengthen the attribution of gains. The public π0.5 checkpoint does not include subtask inference, and Table 13 suggests that System 2 contributes a considerable share of the improvement (48.62 → 57.80 on RoboTwin). A baseline in which π0.5 is driven by the same subtask planner, or by oracle subtasks, would help isolate the contributions of Soft KI and System 3, and a multi-stage training baseline under the same data would directly support the one-stage training claim. In addition, clarifying the configuration behind the TriBrain entry in Table 10 (61.59, which matches row (3) of Table 13, although Appendix D notes that Table 10 excludes System 3), describing how the baselines were adapted to Maker H01, and reporting confidence intervals for the real-robot results would make the evidence easier to interpret. A slightly more scoped discussion of long-horizon capability (the evaluated tasks span roughly 2–4 stages) would also be helpful.

Questions for authors
  1. Could the authors discuss TriVLA, π0.7 and RECAP, and clarify how TriBrain differs from them?

  2. Would it be possible to report π0.5 driven by the same subtask planner, or by oracle subtasks, on a subset of the Table 2 tasks?

  3. Could you clarify the configuration behind the TriBrain entry in Table 10, and the α\alpha values used by the shared and capacity-matched baselines?

  4. How are the per-step value targets derived from trajectory-level completion annotations, and what Δ\Delta is used? Have you compared imagined-future progress labels with labels computed from real trajectories?

  5. Could you briefly describe how the baselines were adapted to Maker H01, and provide confidence intervals for the real-robot results? Since PtP_t is fixed to 1 at deployment, could System 3's value estimates also be used online, for example for failure detection?

3 comments

Response to All Reviewers: We thank reviewers for recognizing our coherent, reasonable three-system framework [EZzK, q2Uq], substantial scale and broad evaluation [EZzK, gFvy, q2Uq], clear, practical real-robot gains [gFvy], and informative component analyses and ablations [gFvy, q2Uq].

W1&Q1: Related works.

First, unlike TriVLA (focused on visual trajectory synthesis), TriBrain's System 3 couples prospective visual subgoals with temporally aligned progress evaluation for dual-signal continuous control. Second, unlike π0.7\pi_{0.7}'s external prompt conditioning, TriBrain autonomously predicts progress directives Pt\mathcal{P}_t from imagined future states and uses a fixed prior (Pt=1\mathcal{P}_t=1) at inference to break execution deadlocks. Third, while RECAP uses binarized value-derived advantages, TriBrain applies asymmetric independent dropout during post-training and asynchronous visual grounding, mitigating representation collapse for robust closed-loop execution.

W2&Q2: Subtask planner for π0.5\pi_{0.5}.

We equipped π0.5\pi_{0.5} with TriBrain's SYSTEM 2 and evaluated three multi-stage tasks on AgileX PiPER and Maker H01 under Table 2's settings:

RobotMethodFood Prep Heating(%)Cube Sorting(%)Dish Cleanup Washing(%)
PiPERπ0.5\pi_{0.5}50.095.060.0
π0.5\pi_{0.5} + Planner70.085.080.0
H01π0.5\pi_{0.5}50.08.340.0
π0.5\pi_{0.5} + Planner65.015.060.0

W2&Q3: Tables 10 & 13 configurations.

Full configurations are detailed in Appendix D. Specifically: (1) Shared 3B (3.5B) uses αKI=0.1\alpha_{KI}=0.1 not 0.01; (2) Capacity-matched shared (6.8B) pairs a 7B-class backbone (~6.3B non-embedding params) with a 0.5B Action Expert using αKI=0.1\alpha_{KI}=0.1; and (3) TriBrain (6.5B) decouples System 2 (3.0B) and System 1 (3.5B) backbones, also with αKI=0.1\alpha_{KI}=0.1. We also correct Table 13(3) from 61.59% to 65.59%. Thank you for catching this oversight.

W2&Q4: Per-step value targets.

During training, the Value Head takes multimodal state inputs (visual observations and language instructions) supervised by trajectory annotations, estimating the state's expected task-completion VtV_t. SYSTEM 3 also predicts a future observation to form an imagined future state, which the Value Head evaluates identically to obtain Vt+ΔV_{t+\Delta}, their difference yields the binary condition (Eq. 3).

W2&Q4: Imagined-future vs. real-trajectory labels.

TriBrain_RT uses real-trajectory annotations. Both fix Pt=1\mathcal{P}_t=1 at inference to isolate supervision sources:

RobotMethodCloth Folding(%)Dish Cleanup(%)
PiPERTriBrain_RT7590
TriBrain80100
H01TriBrain_RT5085
TriBrain80100

These gains highlight the value of imagined-future guidance. Because manipulation involves dynamically evolving object states, contacts, and disturbances, an action's true utility depends on subsequent states. Unlike real-trajectory labels—bounded by local demonstration coverage—comparing current and imagined-future values provides predictive guidance on task advancement.

W2: One-stage vs. Multi-stage.

Aligned with Fig. 3(c) (3000h data), we trained a controlled multi-stage baseline (Fig. 1(a)) under identical settings:

Training ParadigmFold ClothesPick Fruits
Multi-stage (Fig. 1a)10%55%
One-stage(ours)10%65%

These results show that our one-stage paradigm maintains or improves task performance. Furthermore, bypassing the separate VLM and action-expert training significantly reduces training time and resource overhead.

W2&Q5: H01 adaptation & confidence intervals.

Per Appendix B.2, states and actions share a unified joint/end-effector representation, using embodiment-specific binary masks to exclude invalid dimensions for PiPER (6 DoF) and H01 (7 DoF). Using PiPER's statistical method, H01's 95% confidence interval is ±4.1. Moreover, H01's fisheye camera lacks dedicated distortion correction or viewpoint adaptation, likely degrading visual features and spatial localization, especially in Cube Sorting.

Q5: Online failure detection with SYSTEM 3.

At deployment, fixed Pt=1\mathcal{P}_t=1 acts as a forward prior to break action loops (Appendix F; Table 12(b)). SYSTEM 3's online value estimates and progress margins could support real-time failure detection and recovery: sustained value drops or abnormal margins can warn of failure and trigger a recovery policy. Recent work explores World Value Models for execution evaluation and failure detection, a direction we are actively investigating with preliminary progress.

We thank the reviewers again for helping improve our paper's clarity and rigor.

Thank you for the thoughtful response and the additional experiments, which address several of my concerns, though I would still like to see how well the gains hold across a broader range of tasks, so my overall assessment and rating remain unchanged.

Thank you for your thoughtful feedback. We appreciate your concern regarding whether the performance gains can be sustained across a broader range of tasks. We would like to clarify that our evaluation already covers a relatively diverse set of tasks and experimental settings, which may not have been sufficiently highlighted in our initial response.

Specifically, Figure 4 evaluates the base model on 6 tasks, including 3 in-domain and 3 out-of-domain tasks, providing evidence of generalization beyond the training task distribution. Furthermore, Tables 1 and 2report real-world experiments covering 6 and 7 tasks, respectively. Table 3 provides additional evaluation across 3 simulation benchmarks. Importantly, these evaluations involve two distinct robot embodiments, PiPER and H01.

Taken together, our experiments cover in-domain and out-of-domain tasks, real-world deployment, and multiple simulation benchmarks across two robot embodiments. We believe this provides substantial evidence that the observed improvements are not limited to a narrow set of tasks or a single embodiment.

We acknowledge that evaluating on an even broader range of tasks would be valuable future work. Given that the rebuttal period is about to conclude, there is unfortunately insufficient time to conduct additional experiments and report their results rigorously within the current discussion window. We therefore sincerely hope that the reviewer can take the comprehensive experimental evidence already provided in our submission into consideration when assessing the breadth and generalizability of our approach.

Thank you again for your thoughtful feedback and consideration.

Reviewer_EZzKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents TriBrain, a large multi-embodiment system consisting of an action policy, a semantic subtask planner, and a World Value Model that predicts visual subgoals and progress signals. It further introduces Soft Knowledge Insulation, which scales action gradients entering the VLM backbone. Trained on more than 37,000 hours of embodied data and 272 million vision-language samples, the system reports results across two real robots and several simulation and embodied-understanding benchmarks.

Critical strengths
  • The idea of using three systems is reasonable.

  • The training and evaluation effort is substantial.

Critical weaknesses
  • The distinction between SYSTEM 3 and recent world-model-augmented policies such as π0.7\pi_{0.7} is insufficiently established. Both use generated visual subgoals and performance- or progress-related conditioning, while no direct comparison isolates the benefit of TriBrain’s value-difference-based binary labels. Moreover, PtP_t is fixed to 1 during deployment, weakening the claimed role of online progress evaluation.

  • The real-world out-of-the-box OOD evaluation is not fully convincing. Figure 4 reports only TriBrain, without comparisons to π0.5\pi_{0.5} or other pretrained baselines, and the results are highly mixed across tasks and embodiments, including several zero-success cases.

Questions for authors
  • The proposed method appears to be primarily an integration of several existing ideas, including a dual-stream VLA, a separate semantic subtask planner, knowledge insulation, and world-model-based subgoal and progress guidance. It remains unclear which component constitutes the main conceptual novelty.

  • Although the paper provides limited data-scaling ablations, the headline comparisons remain heavily confounded by the 37,000-hour embodied corpus and 272 million vision-language samples. Public baselines are not pretrained on the same data or compute budget, making it difficult to attribute the improvements to the proposed architecture.

  • The introduction introduces several mathematical symbols before formally defining them, which makes the high-level presentation harder to follow.

1 comment

Response to All Reviewers: We thank reviewers for recognizing our coherent, reasonable three-system framework [EZzK, q2Uq], substantial scale and broad evaluation [EZzK, gFvy, q2Uq], clear, practical real-robot gains [gFvy], and informative component analyses and ablations [gFvy, q2Uq].

W1(a): SYSTEM 3 vs. world-model-augmented policies (e.g., π0.7\pi_{0.7}).

Both predict visual subgoals and condition policies, but construct progress guidance differently. π0.7\pi_{0.7} uses subgoals and rich prompts with human-labeled execution quality, failures, and speed. Unlike metadata, Pt\mathcal{P}_t models temporal task-progress direction based on imaged-future without human labels; it provides trajectory-aligned positive/negative training advantage and a fixed-1 deployment prior.

W1(b): Why fix Pt=1\mathcal{P}_t = 1 at deployment?

Online progress evaluation during training and a fixed inference prior are not contradictory. During training, the policy learns both values: 1 denotes a transition contributing to progress; 0 denotes no positive progress. At inference, we fix Pt=1\mathcal{P}_t=1 as a positive prior for continuous task progress and do not feed real-time negative progress predictions to the policy. Repeatedly supplying 0 when stuck could reinforce repetitive behavior and impede escape from action loops. Meanwhile, SYSTEM 3 continues updating visual subgoals to guide the policy.

Table 12(b) compares success rate/average execution time using the same checkpoint: fixed 1, 78.3%/74.8 s; online predicted P^t\hat{\mathcal{P}}_t, 75.0%/91.2 s; Fixed 1 is best on both metrics and is our deployment default. Online value differences distinguish progress from stagnation during training; the positive prior encourages completion. They serve policy learning and task execution, respectively.

W2: The queston about OOD evaluation.

Figure 4 serves as an exploratory characterization of TriBrain's zero-shot generalization and boundaries without task-specific fine-tuning. Direct out-of-the-box deployment of pretrained baselines (e.g., π0.5\pi_{0.5}) on our physical platforms (PiPER and H01) is confounded by two factors: (1) substantial embodiment and sensor distribution shifts (e.g., H01's 7-DoF kinematics and uncorrected fisheye optics), which induce severe perception-action misalignment without target-domain adaptation and thus obscure intrinsic policy capability; and (2) undisclosed pretraining data mixtures, which preclude verifying true OOD boundaries. Consequently, Tables 1–2 establish the controlled benchmark under identical hardware and downstream supervision. We will clarify this scope and refine our cross-embodiment claims in the revision.Additionally, zero-success cases are not mere failures, is a realistic outcome. (Sec. 4.4)These failure modes highlight tactile/force sensing and higher-fidelity physical modeling as key future directions.

Q1: TriBrain's components.

TriBrain's core contribution unifies semantic understanding, continuous action generation, and execution-progress evaluation through one architecture and training mechanism. SYSTEMS 1 and 2 are jointly pretrained in one stage; Soft Knowledge Insulation (Soft KI) enables joint semantic/motor adaptation while preventing action gradients from damaging pretrained semantics. Independently pretrained SYSTEM 3 adds future visual subgoals and progress signals during policy post-training, evaluating actual progress from imagined futures to avoid stalling and unverified execution. This reduces stagnation and repetition in long-horizon manipulation and forms a unified multi-embodiment foundation model.

Q2: Data and compute budgets.

Strict pretraining-resource matching is infeasible, as public baselines vary in data/compute and provide incomplete disclosure. (1) For fair downstream comparison, Tables 1–3 evaluate official checkpoints under identical datasets, metrics, and settings. (2) To isolate architectural gains, Table 10 (Appendix D) uses a capacity-matched shared backbone, showing that TriBrain retains its gains with comparable parameters and data. Finally, Figure 3 ablates pretraining data scale and diversity.

Q3: Question about symbols.

We will clearly define every symbol, including Gt\mathcal{G}_t and Pt\mathcal{P}_t, when first introduced in Sec. 3.

We thank the reviewer again for the insightful feedback and we will revise our paper accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.