acceptodds
Withdrawn from ICLR 2027

RoboICL: Embodied In-Context Learning with GPT-6 Astra

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

28 views

Abstract

In-context learning enables LLMs to adapt to new tasks from demonstrations without updating model parameters. We study whether the same principle can support robot control, where demonstrations must connect language and vision to continuous actions and subsequent decisions must account for execution outcomes. provides a general-purpose multimodal starting point, but its zero-shot performance varies substantially across manipulation tasks. We introduce RoboICL, a framework for closed-loop control that combines a static prefix of compressed expert demonstrations with a bounded dynamic context of the robot's own interactions. Both contexts use the same observation–action call–execution receipt–next observation format, allowing the model to distinguish proposed actions from those actually executed. Bounded memory retains temporally distributed interactions and recent feedback while marking omitted intervals. At each decision round, predicts Cartesian end-effector action blocks from the current observation and retained context, without task-specific model updates or an auxiliary learned action policy. In RoboDojo simulations, adding demonstrations is associated with gains on the evaluated shot-study tasks. Across three single-arm real-robot tasks, mean normalized progress rises from 14.45 at zero shot to 78.89 with three demonstrations. A separate exploratory study coupling JEV with RoboICL substantially improves inference speed on RoboDojo while preserving control capability. These results show that demonstrations and execution-grounded interaction history can serve as an effective inference-time interface for direct robot control.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Multimodal Models, In-Context-Learning, Bimanual Manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WWdFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RoboICL controls robots with a frozen GPT-6 Astra model. It gives the model expert demonstrations plus a bounded history of its own past interactions. The paper reports better progress scores with demonstrations, both in simulation and on three real-robot tasks. It also tries to cut down on model calls by having a separate decision model decide when to reuse predicted actions.

Critical strengths

The idea of using agent with demonstrations for in-context robotic learning makes sense.

Critical weaknesses

1.The novelty is limited. Demonstration prompting, online interaction history, and execution feedback are already common in robot ICL and agentic control. RoboICL combines these with bounded memory and a shared interaction format, but it is unclear what new capability this combination brings. Without comparisons to closely related methods (ICRT, RoboTTT or other agentic frameworks like CaP-X) or ablations showing that the proposed context design matters, the paper reads more as an application of GPT-6 Astra to robot control. 2. There are different components of the proposed method such as demonstrations, online history, execution receipts, temporal anchors, and gap markers. But the importance of each component is not studied. Comparisons against recent-history-only memory, demonstrations without online history, and history without receipts would make the method easier to evaluate.

Questions for authors
  1. Does increasing demonstration count help when the total image and token budget is fixed?
  2. Can you report complete-task success alongside progress scores for every real-robot condition?
  3. How does RoboICL compare with related methods such as ICRT, RoboTTT, and CaP-X.
Reviewer_A4nZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RoboICL uses GPT-6 Astra and upto 1 demonstration in the context to predict caretsian end effector positions for robot manipulation. RoboICL calls the combination of these two things the static prefix and the Dynamic online context. The paper also includes a separate digression into using Jev along with GPT-6-Astra. RoboICL is evaluated on simulated tasks in robodojo and in some real tasks.

Critical strengths
  • The paper is well written and easy to follow.

  • The idea to use a demonstration in the context along with the execution receipts with an off-the-shelf VLM as policy is wonderful. It shows how far VLMs like Astra have come for robotics that we dont need to train VLAs to do the same thing anymore. The equivalent performance of 1-shot Astra and Astra+pi0.5 is impressive. The paper is also candid about its weaknesses.

Critical weaknesses
  • While the paper is candid about its weaknesses, I am not certain it is ready yet for publication because it is missing (as it candidly says), a proper statistical comparison across many seeds and many initial positions of objects in the sim/real world using the same protocol across methods.

  • I am also unclear why there is a missing analysis on more than 1 demonstration in the context, on other VLMs (sort of making this an evaluation benchmark rather than just a 1-demo one-off prompt), on the relationship between the right kind of demonstration for the context (especially because there is just one demo and the readers would love to know what demo they should put inside), etc

Questions for authors

Please see Weaknesses

Reviewer_c3hAOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes RoboICL, an embodied in-context learning framework that uses GPT-6 Astra for robot control without task-specific model training. The framework combines static expert demonstrations with dynamic online interaction history, using a unified observation-action-execution feedback format. It also introduces a bounded memory mechanism to retain selected interactions during long-horizon tasks. Experiments on RoboDojo and three real-robot tasks show promising improvements with additional demonstrations.

Critical strengths

The paper is very up to date and includes recent developments such as GPT-6 Astra, Jev, and RoboDojo. I am impressed that the authors incorporated research released as recently as mid-September 2026. I also like that the authors include real-world robot demos.

Critical weaknesses

My first concern is the limited novelty of the proposed method. Several closely related works have already explored using frozen general-purpose VLMs for robot control with demonstrations, online interaction history, and execution feedback. The main differences in RoboICL seem to be the action representation, explicit execution receipts, and bounded temporal-anchor memory. I am not fully convinced that these differences provide sufficient methodological novelty for ICLR, especially without experiments showing their specific benefits.

Another major concern is the lack of controlled ablation studies. The authors propose several components, including static demonstration context, dynamic online memory, execution receipts, and temporal-anchor selection. However, the current experiments do not clearly isolate the contribution of each component. For example, I would like to see comparisons between the proposed memory mechanism and simply retaining the most recent interactions, or between using execution receipts and not using them. Figure 3 shows different context allocations, but some results come from historical configurations with different settings. I think these experiments are insufficient to validate the proposed design choices.

I also have concerns about the fairness and reliability of the experimental comparisons. In Table 1, the baseline results are taken from previous papers with different evaluation protocols, while RoboICL uses only five layouts per task. The authors also mention that some RoboICL results come from selected historical batches. I am not sure how much of the reported improvement comes from the proposed method rather than differences in evaluation settings. The lack of repeated evaluations and uncertainty estimates further makes it less convincing.

The real-robot experiments are promising, but I think the evaluation is still quite limited, with only three tasks and five trials per setting. The claim about improved inference efficiency with Jev seems somewhat overstated. The evaluation covers only two tasks with five layouts each, and the Jev configuration was selected using the same evaluation layouts. In one task, the number of successful episodes even decreased. I do not think these results are sufficient to support a general claim about improved inference efficiency while preserving control capability.

Questions for authors
  1. How are the expert demonstrations selected for the RoboDojo evaluations? Are the demonstration trajectories collected from different layouts than those used for testing? I would like the authors to clarify this to better understand the evaluation protocol.

  2. In Section 3, the action interface uses a 14-dimensional bimanual action representation with three camera views, while the real-robot experiments use a single Franka arm with two cameras. I would like to know how the interface is adapted to this setting, and how the GELLO teleoperation demonstrations are converted into the same action–receipt format.

  3. Since each task in Table 1 is evaluated on only five layouts, I would like to know the per-episode scores and the variance for Tables 1 and 2, as well as how the RoboICL entries were selected from the historical batches.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.