acceptodds
Under review as a conference paper at ICLR 2027

TEAM-VLA Agents: Chain-of-Thought Reasoning for Multi-Agent Embodied Autonomy with Closed-Loop Recovery

Abstract

For Vision-Language-Action (VLA) based embodied robots, it is essential to establish an autonomous pipeline in open environments, which spans the entire process from goal-driven planning to execution verification and fault recovery. Existing methods often couple high-level planning with reactive execution, lacking explicit reasoning and closed-loop recovery. To address these limitations, we propose the TEAM-VLA Agents (TVA) framework, which adopts a novel paradigm that integrates explicit Chain-of-Thought (CoT) reasoning with multi-agent team collaboration. By leveraging role-based task decomposition and a context-sharing architecture, TVA constructs a CoT-driven autonomous system that unifies task planning, execution monitoring, as well as autonomous failure diagnosis and recovery within a single closed-loop process. This design not only ensures interpretable and traceable task execution but also effectively mitigates error cascades. Empirical evaluations demonstrate that TVA substantially improves success rates on long-horizon tasks and exhibits enhanced robustness and generalizable recovery capabilities across diverse execution failures. Our findings underscore that explicitly incorporating CoT-based reasoning, task verification, and self-recovery mechanisms is a critical step toward achieving reliable, scalable, and truly autonomous embodied intelligence.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Chain-of-Thought(CoT), Multi-Agent, Vision–Language–Action (VLA), Closed-Loop Recovery, Embodied Autonomy

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_7b4aOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the need for VLA systems to detect failures promptly, recover execution, and maintain task context during long-horizon tasks. However, the current manuscript does not clearly describe the key recovery mechanisms, reports internally inconsistent primary CALVIN metrics, and does not adequately specify the recovery budget, training configurations, or the scope of skill-library updates during evaluation. Consequently, the current version does not sufficiently support its full claims regarding a new paradigm, generalizable recovery, and skill expansion.

Critical strengths
  1. The paper addresses an important practical problem: VLA systems need to promptly detect intermediate failures, recover execution, and maintain task context during long-horizon manipulation.

  2. The performance improvements across multiple VLA backbones suggest that the framework may be broadly applicable. The larger gains on long-horizon tasks and weaker backbones are particularly encouraging.

  3. The component ablations, retry-only baseline comparison, and verification-frequency analysis provide useful evidence for understanding the framework’s effectiveness and computational trade-offs.

Critical weaknesses
  1. The principal CALVIN metrics are internally inconsistent. According to the definition in Appendix B.1, average chain length should equal the sum of the success probabilities for completing at least one through five consecutive instructions. However, the TVA success rates in Table 2 imply average chain lengths of 4.307 and 4.479, while the reported values are 4.41 and 4.59. Table 4 also reports 4.59 for ABC→D, inconsistent with Table 2. These discrepancies exceed what rounding can explain and weaken confidence in the quantitative conclusions.

  2. The key recovery mechanisms are insufficiently described. The manuscript does not specify the implementation, training data, or training objective of the future recovery-trajectory predictor. Tracking historical motion alone does not explain how future trajectories conditioned on a recovery goal are generated. Furthermore, the skill-novelty function, threshold, retrieval procedure, and reuse mechanism lack concrete definitions.

  3. The evaluation protocol does not sufficiently distinguish the sources of the performance gains. Recovery-attempt limits, action budgets, model checkpoint configurations, and skill-library update boundaries are unclear. Consequently, it is difficult to determine whether the improvements arise from better recovery decisions, additional execution opportunities, or experience accumulated during testing. The absence of uncertainty estimates also limits the interpretation of smaller performance differences.

  4. The contributions of explicit CoT and role specialization have not been clearly established. The configuration of “w/o CoT Planner” is insufficiently specified and may conflate removing explicit reasoning with removing task decomposition. The existing evidence does not clearly establish what distinctive contributions the framework offers beyond a conventional planning–verification–replanning pipeline with shared memory.

Questions for authors
  1. What model or algorithm generates the future recovery trajectories? What training data and objectives are used? How are the visual points selected?

  2. Do Full TVA and the retry-only baseline use the same model checkpoints, action budgets, timeouts, and maximum number of recovery attempts? Is the skill library reset between evaluation chains, or continuously updated throughout testing?

  3. What functionality is retained in the “w/o CoT Planner” ablation? Does it preserve the same model, structured task decomposition, observation inputs, and output interface?

Reviewer_HnidOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes the TEAM-VLA Agents multi-agent closed-loop framework, which relies on the reasoning of thought chains, role-division intelligent agents, and shared context memory. Without modifying the underlying VLA model, it effectively alleviates the cascading error problem in long-term sequence robot manipulation tasks, and improves the success rate and robustness of the tasks.

Critical strengths
  1. The proposed method is modular and plug-in capable, reusing the existing VLA base and enhancing capabilities through the upper-level multi-agent system.
  2. The proposed method has been verified to be capable of autonomous fault recovery and suppressing cascading errors in long-time sequence tasks.
Critical weaknesses
  1. The paper only analyzes the time cost brought by the monitoring and verification process, but does not provide a direct comparison of the total inference time of the complete end-to-end system and the baseline VLA.
  2. The autonomous fault recovery capability is still limited. When facing new and unencountered fault patterns, the system lacks sufficient generalization and repair capabilities.
  3. Please explain the differences between the proposed method and the agentic VLA work such as harnessvla, and compare it with previous works.
Questions for authors

See the above weaknesses.

Reviewer_eq7fOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes TEAM-VLA Agents (TVA), a closed-loop wrapper around existing VLA policies for long-horizon manipulation. A Planner (Qwen3-VL-8B-Thinking) breaks the instruction into ⟨skill, object, goal⟩ subtasks, a VLA Executor (π0.5, OpenVLA-OFT, or SmolVLA) carries them out, and a Monitor (Qwen3-VL-8B-Thinking) labels each subtask Running/Success/Failure with a failure cause; all three share a context memory. On failure, the Planner writes a recovery subtask, a multi-point tracking model predicts image-space tracks that are drawn onto the observation to guide the VLA, and successful novel recoveries are added to a skill library. Evaluation covers LIBERO (three backbones), CALVIN (π0.5), ablations, recovery and detection rates, and qualitative real-robot examples.

Critical strengths
  • Consistent gains on long-horizon tasks, where errors compound, across VLA backbones. On CALVIN, five-in-a-row success rises from 64.3% to 75.2% (ABC→D) and from 68.7% to 78.3% (D→D), and the gap widens with chain length (Table 2, Fig. 1). On LIBERO, all three backbones gain most on the Long suite (e.g., OpenVLA-OFT 86.5 → 90.4, SmolVLA 63.3 → 71.5; Table 1). The pattern fits the paper's error-propagation motivation, and wrapping both autoregressive and flow-matching policies is practically useful.
  • Recovery helps beyond simple retrying. With the planner and monitor held fixed, full TVA beats the retry-only monitor on both LIBERO-Long (83.6 → 87.3) and CALVIN average chain length (4.31 → 4.59; Table 4). This is the clearest evidence that the recovery mechanism, not just extra attempts, contributes.
Critical weaknesses
  • The method is under-specified, and the released code does not match the paper, so the main numbers cannot be verified. The paper does not name the multi-point tracking model F_track (Eq. 7; absent from Table 7) or define the novelty measure ν(σ) and threshold δ_n (Eq. 9). Sec. 3.3 states that TVA "does not alter the architecture or training objective of the underlying VLA model", yet Eq. 3 adds a context feature c_t, Eq. 8 conditions the policy on track-overlay images, and Fig. 2(d) shows "Supervised / RL Fine-Tuning". The anonymous repository does not resolve this: it calls itself "an integration blueprint, not an already trained experiment", fills the gaps with its own choices (e.g., "Paper Eq. 7-8 leave the tracking architecture unspecified"), and tags every trajectory predictor "original_paper_model": False. It also contains a full π0.5 fine-tuning config on a recovery dataset (configs/training/pi05_recovery.yaml), and it saves the skill library to a file path set in the provided config, so recovery skills learned in one evaluation episode carry over to later ones. It is therefore unclear whether the LIBERO/CALVIN gains come from the closed loop, from extra VLA fine-tuning, or from adapting on test episodes. Releasing the code and checkpoints behind Tables 1–6, specifying the tracker and any fine-tuning (data, steps), and reporting results with the library reset per episode would address this.
  • Several claims go beyond the evidence. Contribution 3 claims "superior generalization on multi-object scenarios and complex linguistic instructions", but no experiment targets either. The claim of "generalizable recovery capabilities across diverse execution failures" rests only on aggregate recovery rates (39.7–44.5% for π0.5, Table 5), with no breakdown by failure type. "SOTA performance" is claimed without stating whether baseline numbers were reproduced under the same protocol. Table 4 is described as "LIBERO-Long success increasing from 76.8% to 87.3%", but its first row (π0.5 with a naive planner, 76.8) is 8 points below vanilla π0.5 (84.8, Table 1); the end-to-end gain over the backbone is +2.5. The claims should be narrowed to what is tested, or the missing experiments added.
Questions for authors
  1. [Clarification] Were the baseline numbers in Tables 1–2 reproduced under your protocol or taken from the original papers?
  2. [Clarification] Why do the Naive Planner (76.8) and CoT Planner (80.5) variants in Table 4 fall below vanilla π0.5 (84.8)? What replaces the planner in Table 3's "w/o CoT Planner" row? The released ablation code has no such variant.
  3. [Clarification] In the released code, select_visual_points requires pixel annotations that the LIBERO wrapper does not provide, and otherwise falls back to fixed screen positions. How were the visual-trace points obtained in the reported experiments?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.