- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents TRIBRAIN, a multi-embodiment vision-language-action framework designed to jointly address semantic understanding, continuous action generation, and predictive execution guidance. The model consists of three functional systems: SYSTEM 1 generates continuous robot actions, SYSTEM 2 performs task understanding and subtask decomposition, and SYSTEM 3 predicts future visual subgoals and estimates temporal task progress. To facilitate joint vision-language-action learning, the paper introduces Soft Knowledge Insulation, which attenuates action-loss gradients entering the vision-language backbone. During policy post-training, the frozen World Value Model in SYSTEM 3 provides visual subgoal and progress-aware guidance to the action policy, with independent dropout used to improve robustness. The paper conducts extensive experiments across foundation-model evaluation, real-robot post-training, simulation benchmarks, and embodied spatial/affordance understanding. Results are reported on multiple robot embodiments, including AgileX PiPER and Maker H01, as well as benchmarks such as RoboTwin 2.0, EBench, RoboColiseum, and MiMo-Embodied. The large-scale evaluation, together with ablations on architectural decoupling, Soft KI, visual subgoals, and progress guidance, provides broad empirical evidence for the proposed framework.
Critical strengths
- The paper proposes a coherent three-system VLA framework that separates task understanding, action generation, and future-state prediction with progress evaluation. This decomposition addresses important challenges in long-horizon, multi-embodiment robot control.
- The evaluation is broad and substantial, covering large-scale pretraining, multiple robot embodiments, real-world manipulation, simulation benchmarks, and embodied spatial and affordance understanding. This breadth makes the empirical claims more consequential than results on a single robot or benchmark.
- The paper includes informative ablations of architectural decoupling, Soft Knowledge Insulation, visual subgoals, and progress guidance. These experiments provide evidence that the reported improvements are associated with specific design components rather than only with the overall model size.
Critical weaknesses
- TRIBRAIN uses over 37,000 trajectory hours and 272M VL samples, which may be substantially more than the baselines. The paper should clarify the pretraining data and compute budgets, and provide matched comparisons to determine whether the gains arise from the proposed design or simply from larger-scale training.
- In SYSTEM 3, the binary progress signal is defined by whether the predicted future value is strictly greater than the current value. For trajectories that are close to completion, small prediction errors could cause the progress labels at adjacent time steps to alternate between 0 and 1. The paper should clarify whether it uses a margin, temporal smoothing, hysteresis, or another stabilization mechanism.
- For each benchmark, the paper should specify the pretrained checkpoints, post-training data, and adaptation protocol used by each method, including whether evaluation is zero-shot or fine-tuned.
- Since PiPER and H01 are separately post-trained with embodiment-specific masks, the current results do not establish embodiment-general control. Zero-shot cross-embodiment transfer or shared-representation analyses would strengthen this claim.
- The experiments are broad and solid, but the paper’s writing does not clearly communicate its main contributions. The roles of the three systems, Soft KI, and dual-signal guidance are not sufficiently distinguished from existing techniques and system-level integration. The paper would benefit from a clearer contribution statement.
Questions for authors
- How do the pretraining data and compute budgets of TRIBRAIN compare with those of each baseline? Are the reported gains preserved under data- and compute-matched comparisons?
- For each benchmark, which pretrained checkpoint, post-training dataset, and adaptation protocol are used for TRIBRAIN and each baseline? Are the results zero-shot, few-shot, or fully fine-tuned?
- In SYSTEM 3, how are per-timestep progress targets constructed, and how is the strict value-difference threshold stabilized against small prediction errors and temporal label flipping?
- Since PiPER and H01 are separately post-trained with embodiment-specific action masks, what evidence shows that TRIBRAIN learns embodiment-general control rather than robot-specific policies? Do you have zero-shot cross-embodiment or shared-representation analyses?
2 comments
Response to Author Rebuttal
Thank you for the detailed responses and additional clarifications. The explanations regarding resource-matched comparisons, evaluation settings, progress target construction, and cross-embodiment transfer help address my questions. The additional evidence further strengthens the empirical support for the proposed framework.
Overall, I appreciate the authors' efforts in addressing my concerns and maintain my positive assessment of the paper.
Response to All Reviewers: We thank reviewers for recognizing our coherent, reasonable three-system framework [EZzK, q2Uq], substantial scale and broad evaluation [EZzK, gFvy, q2Uq], clear, practical real-robot gains [gFvy], and informative component analyses and ablations [gFvy, q2Uq].
W1&Q1: Data- and compute-matched comparisons.
Public information on baseline pretraining data, duration, and compute is inconsistent, so a strict resource-matched comparison of original pretraining may be impossible. We therefore control downstream post-training and conduct capacity-matched experiments to better isolate model-design effects.(1)TriBrain and all baselines in Tables 1–3 use the same downstream datasets and compute, with full-parameter fine-tuning on 8 A100 GPUs. These gains thus do not arise from extra post-training data or compute; they reflect model-design differences under unified post-training conditions.(2)Table 10's capacity-matched experiments further assess model capacity and computational cost. TriBrain retains gains under comparable capacity and cost, further supporting our architectural design.
W3&Q2: Benchmarks and evaluation settings.
TriBrain initializes from our multi-embodiment pretrained checkpoint, while baselines use official pretrained checkpoints and recommended configurations. Specifically: (1)Fig. 4 (real-robot): Evaluates our pretrained TriBrain checkpoint directly on PiPER and H01 without post-training, testing generalization (OOD, zero-shot) and cross-embodiment transfer (ID,few-shot). (2)Tables 1–2 (real-robot, fine-tuned): Uses identical downstream task datasets and full-parameter fine-tuning (8 A100 GPUs, matched compute budgets) (3)Table 3 (simulation, fine-tuned): Fine-tunes all models on each simulation benchmark's official training split under identical observation inputs, camera views, and control frequencies.
Q3: How are per-timestep progress targets constructed.
Per-timestep progress targets are constructed from sparse episodic success/failure labels. For a successful episode ending at terminal step Tend, we define the reward as rt=−1 for all pre-terminal steps and rTend=0 upon completion, whereas failed episodes receive a large negative constant. The empirical return is then discretized and normalized per task into (−1,0) to train the distributional value function.
W2&Q3: Threshold robustness to prediction errors and label flipping.
Through three complementary mechanisms: (1) Temporal windowing: comparing states separated by Δt (aligned with the action chunk) captures macro task advancement rather than reacting to high-frequency, adjacent-frame noise. (2) Continuous value learning: the Value Head regresses continuous expected return targets rather than performing noisy binary classification, providing a smooth underlying representation before discretization. (3) Fixed deployment prior: at inference, we clamp Pt=1 as a constant positive condition, completely decoupling deployment guidance from online prediction jitter or training-time label flips. Finally, Appendix G empirically confirms that these temporal advantage labels improve policy performance despite discretization.
W4&Q4: Embodiment-general control and cross-embodiment transfer.
Figure 4 demonstrates zero-shot cross-embodiment transfer: the pretrained TriBrain checkpoint directly runs on PiPER and H01 without task-specific adaptation, achieving OOD success rates of 84.4%/60.0% (PiPER) and 30.0%/10.0% (H01). This proves that multi-embodiment pretraining learns transferable manipulation knowledge rather than relying solely on robot-specific post-training. Furthermore, embodiment-specific action masks only handle action dimensions and control degrees of freedom during post-training, meaning visual semantics and task representations are shared rather than relearned from scratch for each robot.
We thank the reviewer for helping improve the final manuscript.