- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
META-Robot addresses the problem of general-purpose robotic manipulation with an agentic robotics setup. It extends the agentic robotics work, by enforcing predicate-based skill generation and learning a predicate library. In addition, it continuously learns success criteria checker via proprioceptive signals. The work is tested on Libero (against multiple agentic robotic baselines), Libero-Long (to show zero-shot transfer, with no comparisons) and real-world robot (without comparisons).
Critical strengths
- The approach is intuitive and seems to be quite generalizable, as it allows for more grounded skill execution and scoring.
- Having both a predicate library and skill library is nice, as many times similar categories of predicates are used
- Proprioceptive signal and it leveraging contact and load evidence is very nice as a verifier
- I like the variety of experiments, testing different cases where agentic robotics will be used.
- Has a real-world demonstration of the approach.
Critical weaknesses
- (Evaluation, major) While I appreciate the variety of experiments (Real-robot evaluation, Zero-shot transfer to unseen tasks), it would be important to compare it against at least the highest-performant baseline (ASPIRE w/ GPT-6 Astra), in order to understand the impact of the proposed approach.
- (Evaluation) I feel like this will even be more helpful in longer-horizon tasks, where some agentic robotics systems may get more confused or fail due to lack of predicate reasoning. Potentially, testing this in long-horizon tasks for example in Robocasa will be more fruitful to show benefit over ASPIRE.
- Question: Are there cases where it is hard to identify predicates, in that case, would having this be too challenging and therefore the skill would not execute? A few sentences on this will be helpful.
- (minor) Potentially better names can be made for key modules. Instead of Proprioceptive Signal Orchestration, maybe Proprioceptive-based Criteria and Skill Generation?
Questions for authors
- While I appreciate the variety of experiments (Real-robot evaluation, Zero-shot transfer to unseen tasks), it would be important to compare it against at least the highest-performant baseline (ASPIRE w/ GPT-6 Astra), in order to understand the impact of the proposed approach. How would Meta-Robots compare with ASPIRE w/ GPT-6 Astra in real-robot evaluation and zero-shot transfer? I think zero-shot transfer experiment is a must-have as it is in sim, real-robot comparison is a nice-to-have.
- Is ASPIRE the baseline also with GPT-6 Astra? If not, why is there a difference, and how can you isolate the benefit of the proposed approach?
5 comments
Response to the weakness 2&3&4
W2
Thank you for your comment; your point and intuition are very reasonable, and we fully agree. Indeed, as you noted, the presence of predicates allows the robot system to reason more accurately and reduce confusion, and on long-horizon tasks our framework can demonstrate its advantages more clearly. We therefore tested on RoboCasa365 as you suggested. Due to time constraints, we have not yet been able to reproduce ASPIRE on RoboCasa, but we compare with HarnessVLA using its publicly available code for reproduction, and our results show an advantage on the evaluated RoboCasa task groups. Thank you for your suggestion.
For RoboCasa365, we use the target50 split, comprising 18 Atomic-Seen tasks, 16 Composite-Seen tasks, and 16 Composite-Unseen tasks. For each task, seed 0 is used for development and memory construction and is excluded from evaluation; seeds 1–5 are used for testing. We conduct four experimental runs, yielding 50 × 5 × 4 = 1,000 evaluation episodes per method. The corresponding evaluation denominators are 360 for Atomic-Seen and 320 each for Composite-Seen and Composite-Unseen. Each group-specific success rate is computed from the total number of successful episodes across the four runs divided by its corresponding evaluation denominator. META-Robot / GPT-6 Astra achieves success rates of 94.44% on Atomic-Seen and 86.25% on Composite-Seen.
| Method | Atomic-Seen | Composite-Seen | Composite-Unseen |
|---|---|---|---|
| META-Robot / GPT-6 Astra | 94.44% | 86.25% | 85.00% |
| Harness VLA — Codex / GPT-6 Astra | 87.78% | 43.75% | 42.50% |
| Harness VLA — Claude Opus 4.7 | 80.00% | 47.50% | 15.00% |
W3
Thank you for your question; I will explain it concisely.
- There are cases where predicates are difficult to recognize, for example because the relevant state is occluded or the available observations are ambiguous. The issue you raise is in fact one of the motivations for incorporating proprioceptive signals. For the towel-grasping task, we initially relied only on visual signals, but we could never recognize the predicate for the critical condition of whether the gripper tip is in contact with the towel.
- When a predicate is hard to recognize, an undetermined predicate does not block all actions. In this case, the LLM understands which action and the associated skill should be executed at this step, and it will attempt to execute the skill to complete the corresponding action. For instance, in the situation I just mentioned, the system may invoke the corresponding skill to try to grasp the towel; although the grasp may ultimately fail, a subsequent predicate will record the failure and help locate the root cause in the revision stage for correction. Therefore, predicates actually serve both diagnosis and execution control: insufficient evidence can trigger new observation or active probing; detecting a failure can trigger budget-limited local repair; and the generated trajectories can be used to guide revision of the program or of its observation and verification mechanisms.
W4
This is an excellent suggestion; thank you very much. It conveys more precisely what our text and this module intend to express, highlighting that proprioceptive feedback supports both executable behaviors and the criteria for evaluating those behaviors. We have revised the main text accordingly.
Response to the question 1&2
Q1
Thank you for your question. Above we showed the comparison between our method and ASPIRE using GPT-6 Astra; we also supplement this on real-robot tasks, with META-Robot following the model configuration and experimental settings reported in the original manuscript (note: the ASPIRE code used is the version prior to the ICLR submission deadline).
| Task | ASPIRE / GPT-6 Astra | META-Robot |
|---|---|---|
| Tennis-ball pick-and-place | 8/8 | 8/8 |
| Put-General | 5/8 | 7/8 |
| Towel folding | 0/8 | 5/8 |
| Breakfast preparation | 0/8 | 5/8 |
| Instruction-guided cup stacking | 7/8 | 8/8 |
| Plug insertion | 1/8 | 3/8 |
| Weight comparison | 0/8 | 8/8 |
Additionally, we tested zero-shot transfer of META-Robot from the Tennis-ball pick-and-place task to a cup-stacking task (requiring only one cup-stacking arrangement, without different instructions), achieving a best success rate of 6/8, compared with 4/8 for ASPIRE / GPT-6 Astra in the same zero-shot transfer setting, as shown in the transfer table above. The real-robot tests and the real-robot zero-shot transfer tests show that our framework still exhibits a certain degree of superiority on real robots. However, regarding cross-task transfer, the current experiments support partial zero-shot transfer of already-accumulated knowledge, rather than stable and strong generalization to all unseen tasks; this is closely tied to how well the generated skills match unseen tasks, task similarity, and so on. We will continue to extend research on cross-task transfer in the future.
Q2
Thank you for pointing out this lack of clarity. In the original comparison, the ASPIRE baseline used Claude Opus 4.6, while META-Robot's main results used GPT-6 Astra. We agree that this comparison alone cannot separate the contribution of the framework itself from that of the language model.
To this end, in addition to the META-Robot / GPT-6 Astra versus ASPIRE / GPT-6 Astra comparison above, we also add results for META-Robot / Claude Opus 4.6 versus ASPIRE / Claude Opus 4.6, and we report the base performance of the various LLM models we tested on LIBERO-Pro, as shown in the table below:
| Method | Obj/Task | Obj/Pos | Goal/Task | Goal/Pos | Spatial/Task | Spatial/Pos |
|---|---|---|---|---|---|---|
| META-Robot / Chat GPT-6 Astra | 99.00% | 99.60% | 69.80% | 85.80% | 95.80% | 91.40% |
| ASPIRE / Chat GPT-6 Astra | 98.20% | 99.80% | 67.20% | 85.00% | 81.20% | 79.60% |
| META-Robot / Claude Opus 4.6 | 96.40% | 98.20% | 65.60% | 84.60% | 78.80% | 76.80% |
| META-Robot (DeepSeek-V4-Flash) | 88.60% | 98.00% | 64.60% | 72.20% | 88.40% | 83.00% |
| ASPIRE / Claude Opus 4.6 | 95.00% | 98.00% | 45.00% | 81.00% | 60.00% | 51.00% |
| Gemini 3.1 Pro Preview | 22.00% | 20.40% | 18.00% | 26.00% | 10.00% | 9.00% |
| Claude Opus 4.6 | 20.00% | 9.00% | 16.00% | 38.00% | 2.00% | 15.00% |
| DeepSeek / V4-Flash | 16.00% | 5.00% | 0.00% | 12.80% | 0.00% | 10.00% |
| GPT-6 Astra | 88.60% | 61.00% | 42.00% | 76.60% | 43.80% | 27.60% |
In summary, beyond the addition of predicates — which allow our framework to evaluate not only task execution but also the system itself, i.e., whether the system's judgments about task or skill execution are accurate and effective enough, which is what we mean by needing to extend not only capability but also cognition, and which gives us better performance on some widely used benchmarks — more importantly, our system can understand proprioception, which has enabled progress on tasks such as Weight comparison that other approaches cannot complete at all.
Finally, I really appreciate your comments and questions. We look forward to your reply and hope to work with you on something valuable for the community.
Response to the weakness 1
Thank you for your comments and questions. We are glad to have this opportunity to discuss with you and to offer the community something genuinely interesting and valuable. We respond to each point in turn below. We apologize for the slightly delayed response, as we conducted additional real-robot experiments to supplement our evaluation.
W1
We agree that comparing ASPIRE and META-Robot under the same language model backbone is very important, especially in the zero-shot transfer setting. Thank you for your suggestion; in the table below we add the LIBERO-Pro results for the ASPIRE / GPT-6 Astra version.
Following your suggestion, we tested ASPIRE with GPT-6 Astra; the results are as shown above. As can be seen, its performance remains below our framework. Thank you for this comment — it is a pleasure to exchange ideas with you.
Zero-Shot Comparison
Beyond this, we agree with your judgment that a zero-shot comparison is necessary. We therefore add a comparison experiment that transfers the frozen predicate library and skill library to LIBERO-Pro Long, both using Chat GPT-6 Astra, against the corresponding ASPIRE / GPT-6 Astra baseline:
The zero-shot long-horizon task transfer results show that, with a moderate amount of source-task experience, META-Robot achieves good transfer ability and slightly outperforms ASPIRE: at the 50% accumulation stage, Task and Swap success rates reach 31.8% and 21.2%, respectively, which are 5.8 and 9.2 percentage points higher than ASPIRE; at the 75% stage, Task success further reaches 40.2%, 8.8 points ahead of ASPIRE. To be honest, however, transfer performance depends not only on the amount of accumulated skills but, more importantly, on how well the generated skills and their compositions match the target task's requirements; the lack of consistent gains from further accumulation also suggests that we should further strengthen skill selection and composition targeted at the target task.
We therefore also acknowledge that the current transfer performance is uneven, and our framework's success rate does not increase monotonically with library size. Hence, the current evidence supports partial zero-shot transfer of already-accumulated knowledge, rather than stable and strong generalization to all unseen tasks. We will continue to extend this research on cross-task transfer in the future.
Real-Robot Comparison
For META-Robot, the real-robot experiments follow the model configuration and experimental settings reported in the original manuscript. We evaluated ASPIRE using GPT-6 Astra on the same seven real-robot tasks:
We also add a comparison of the experimental results for ASPIRE with GPT-6 Astra on the real Galaxea R1 Lite robot. We will also clarify the observation and control interfaces available to each method, including access to proprioceptive signals, so as to avoid conflating differences in sensor availability with how each framework uses those signals.
Without task-specific adaptation, we also add, as you requested, an additional real-robot transfer case: we tested transfer from tennis-ball pick-and-place to a fixed-target cup-stacking task (which does not require instruction following — i.e., stacking cups in different orders according to the input color sequence — but only requires stacking the cups in a specific order), as shown in the table below.