acceptodds
Under review as a conference paper at ICLR 2027

META-Robot: Self-Cognition and Capability Scaling in Agentic Robots

10 views

Abstract

LLM-based robotic agent systems can generate programs for robot control, yet their generalization to new tasks often depends on elaborate scaffolding. Expanding their skill repertoires alone offers limited guidance on how to assess new capabilities or identify what remains beyond their reach. We introduce META-Robot, a framework that couples self cognition with capability scaling by expanding the criteria through which an agent evaluates and develops its skills. At its core, action-level predicates make execution outcomes explicit and available for LLM reasoning. The agent uses these assessments to identify its own limitations, extend relevant evaluation criteria, and guide the structured expansion of its skills. Proprioceptive feedback further grounds this process in the robot’s physical state, enriching the evidence available for self-cognition. Experiments in simulation and on real robots demonstrate stronger overall task performance, generalization to new tasks, continued expansion of skills and evaluation criteria, and effective use of non-visual proprioceptive signals

Primary area:
applications to robotics, autonomy, planning
Keywords:
Agentic Robotics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HswaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

META-Robot addresses the problem of general-purpose robotic manipulation with an agentic robotics setup. It extends the agentic robotics work, by enforcing predicate-based skill generation and learning a predicate library. In addition, it continuously learns success criteria checker via proprioceptive signals. The work is tested on Libero (against multiple agentic robotic baselines), Libero-Long (to show zero-shot transfer, with no comparisons) and real-world robot (without comparisons).

Critical strengths
  • The approach is intuitive and seems to be quite generalizable, as it allows for more grounded skill execution and scoring.
  • Having both a predicate library and skill library is nice, as many times similar categories of predicates are used
  • Proprioceptive signal and it leveraging contact and load evidence is very nice as a verifier
  • I like the variety of experiments, testing different cases where agentic robotics will be used.
  • Has a real-world demonstration of the approach.
Critical weaknesses
  • (Evaluation, major) While I appreciate the variety of experiments (Real-robot evaluation, Zero-shot transfer to unseen tasks), it would be important to compare it against at least the highest-performant baseline (ASPIRE w/ GPT-6 Astra), in order to understand the impact of the proposed approach.
  • (Evaluation) I feel like this will even be more helpful in longer-horizon tasks, where some agentic robotics systems may get more confused or fail due to lack of predicate reasoning. Potentially, testing this in long-horizon tasks for example in Robocasa will be more fruitful to show benefit over ASPIRE.
  • Question: Are there cases where it is hard to identify predicates, in that case, would having this be too challenging and therefore the skill would not execute? A few sentences on this will be helpful.
  • (minor) Potentially better names can be made for key modules. Instead of Proprioceptive Signal Orchestration, maybe Proprioceptive-based Criteria and Skill Generation?
Questions for authors
  • While I appreciate the variety of experiments (Real-robot evaluation, Zero-shot transfer to unseen tasks), it would be important to compare it against at least the highest-performant baseline (ASPIRE w/ GPT-6 Astra), in order to understand the impact of the proposed approach. How would Meta-Robots compare with ASPIRE w/ GPT-6 Astra in real-robot evaluation and zero-shot transfer? I think zero-shot transfer experiment is a must-have as it is in sim, real-robot comparison is a nice-to-have.
  • Is ASPIRE the baseline also with GPT-6 Astra? If not, why is there a difference, and how can you isolate the benefit of the proposed approach?
5 comments
AuthorsOpenReview ↗

Response to the weakness 1

Thank you for your comments and questions. We are glad to have this opportunity to discuss with you and to offer the community something genuinely interesting and valuable. We respond to each point in turn below. We apologize for the slightly delayed response, as we conducted additional real-robot experiments to supplement our evaluation.

W1

We agree that comparing ASPIRE and META-Robot under the same language model backbone is very important, especially in the zero-shot transfer setting. Thank you for your suggestion; in the table below we add the LIBERO-Pro results for the ASPIRE / GPT-6 Astra version.

MethodObj/TaskObj/PosGoal/TaskGoal/PosSpatial/TaskSpatial/Pos
ASPIRE / GPT-6 Astra98.20%99.80%67.20%85.00%81.20%79.60%
META-Robot / GPT-6 Astra99.00%99.60%69.80%85.80%95.80%91.40%

Following your suggestion, we tested ASPIRE with GPT-6 Astra; the results are as shown above. As can be seen, its performance remains below our framework. Thank you for this comment — it is a pleasure to exchange ideas with you.

Zero-Shot Comparison

Beyond this, we agree with your judgment that a zero-shot comparison is necessary. We therefore add a comparison experiment that transfers the frozen predicate library and skill library to LIBERO-Pro Long, both using Chat GPT-6 Astra, against the corresponding ASPIRE / GPT-6 Astra baseline:

Method0%25%50%75%100%
META-Robot (Task)0.00%0.00%31.80%40.20%37.00%
META-Robot (Swap)0.00%0.00%21.20%8.00%8.20%
ASPIRE (Task)3.00%12.00%26.00%31.40%37.00%
ASPIRE (Swap)0.00%0.00%12.00%16.60%20.80%

The zero-shot long-horizon task transfer results show that, with a moderate amount of source-task experience, META-Robot achieves good transfer ability and slightly outperforms ASPIRE: at the 50% accumulation stage, Task and Swap success rates reach 31.8% and 21.2%, respectively, which are 5.8 and 9.2 percentage points higher than ASPIRE; at the 75% stage, Task success further reaches 40.2%, 8.8 points ahead of ASPIRE. To be honest, however, transfer performance depends not only on the amount of accumulated skills but, more importantly, on how well the generated skills and their compositions match the target task's requirements; the lack of consistent gains from further accumulation also suggests that we should further strengthen skill selection and composition targeted at the target task.

We therefore also acknowledge that the current transfer performance is uneven, and our framework's success rate does not increase monotonically with library size. Hence, the current evidence supports partial zero-shot transfer of already-accumulated knowledge, rather than stable and strong generalization to all unseen tasks. We will continue to extend this research on cross-task transfer in the future.

Real-Robot Comparison

For META-Robot, the real-robot experiments follow the model configuration and experimental settings reported in the original manuscript. We evaluated ASPIRE using GPT-6 Astra on the same seven real-robot tasks:

TaskASPIRE / GPT-6 AstraMETA-Robot
Tennis-ball pick-and-place8/88/8
Put-General5/87/8
Towel folding0/85/8
Breakfast preparation0/85/8
Instruction-guided cup stacking7/88/8
Plug insertion1/83/8
Weight comparison0/88/8

We also add a comparison of the experimental results for ASPIRE with GPT-6 Astra on the real Galaxea R1 Lite robot. We will also clarify the observation and control interfaces available to each method, including access to proprioceptive signals, so as to avoid conflating differences in sensor availability with how each framework uses those signals.

Without task-specific adaptation, we also add, as you requested, an additional real-robot transfer case: we tested transfer from tennis-ball pick-and-place to a fixed-target cup-stacking task (which does not require instruction following — i.e., stacking cups in different orders according to the input color sequence — but only requires stacking the cups in a specific order), as shown in the table below.

Method0%25%50%75%100%
META-Robot0/80/83/86/86/8
ASPIRE / GPT-6 Astra0/80/81/83/84/8
AuthorsOpenReview ↗

Response to the weakness 2&3&4

W2

Thank you for your comment; your point and intuition are very reasonable, and we fully agree. Indeed, as you noted, the presence of predicates allows the robot system to reason more accurately and reduce confusion, and on long-horizon tasks our framework can demonstrate its advantages more clearly. We therefore tested on RoboCasa365 as you suggested. Due to time constraints, we have not yet been able to reproduce ASPIRE on RoboCasa, but we compare with HarnessVLA using its publicly available code for reproduction, and our results show an advantage on the evaluated RoboCasa task groups. Thank you for your suggestion.

For RoboCasa365, we use the target50 split, comprising 18 Atomic-Seen tasks, 16 Composite-Seen tasks, and 16 Composite-Unseen tasks. For each task, seed 0 is used for development and memory construction and is excluded from evaluation; seeds 1–5 are used for testing. We conduct four experimental runs, yielding 50 × 5 × 4 = 1,000 evaluation episodes per method. The corresponding evaluation denominators are 360 for Atomic-Seen and 320 each for Composite-Seen and Composite-Unseen. Each group-specific success rate is computed from the total number of successful episodes across the four runs divided by its corresponding evaluation denominator. META-Robot / GPT-6 Astra achieves success rates of 94.44% on Atomic-Seen and 86.25% on Composite-Seen.

MethodAtomic-SeenComposite-SeenComposite-Unseen
META-Robot / GPT-6 Astra94.44%86.25%85.00%
Harness VLA — Codex / GPT-6 Astra87.78%43.75%42.50%
Harness VLA — Claude Opus 4.780.00%47.50%15.00%

W3

Thank you for your question; I will explain it concisely.

  • There are cases where predicates are difficult to recognize, for example because the relevant state is occluded or the available observations are ambiguous. The issue you raise is in fact one of the motivations for incorporating proprioceptive signals. For the towel-grasping task, we initially relied only on visual signals, but we could never recognize the predicate for the critical condition of whether the gripper tip is in contact with the towel.
  • When a predicate is hard to recognize, an undetermined predicate does not block all actions. In this case, the LLM understands which action and the associated skill should be executed at this step, and it will attempt to execute the skill to complete the corresponding action. For instance, in the situation I just mentioned, the system may invoke the corresponding skill to try to grasp the towel; although the grasp may ultimately fail, a subsequent predicate will record the failure and help locate the root cause in the revision stage for correction. Therefore, predicates actually serve both diagnosis and execution control: insufficient evidence can trigger new observation or active probing; detecting a failure can trigger budget-limited local repair; and the generated trajectories can be used to guide revision of the program or of its observation and verification mechanisms.

W4

This is an excellent suggestion; thank you very much. It conveys more precisely what our text and this module intend to express, highlighting that proprioceptive feedback supports both executable behaviors and the criteria for evaluating those behaviors. We have revised the main text accordingly.

AuthorsOpenReview ↗

Response to the question 1&2

Q1

Thank you for your question. Above we showed the comparison between our method and ASPIRE using GPT-6 Astra; we also supplement this on real-robot tasks, with META-Robot following the model configuration and experimental settings reported in the original manuscript (note: the ASPIRE code used is the version prior to the ICLR submission deadline).

TaskASPIRE / GPT-6 AstraMETA-Robot
Tennis-ball pick-and-place8/88/8
Put-General5/87/8
Towel folding0/85/8
Breakfast preparation0/85/8
Instruction-guided cup stacking7/88/8
Plug insertion1/83/8
Weight comparison0/88/8

Additionally, we tested zero-shot transfer of META-Robot from the Tennis-ball pick-and-place task to a cup-stacking task (requiring only one cup-stacking arrangement, without different instructions), achieving a best success rate of 6/8, compared with 4/8 for ASPIRE / GPT-6 Astra in the same zero-shot transfer setting, as shown in the transfer table above. The real-robot tests and the real-robot zero-shot transfer tests show that our framework still exhibits a certain degree of superiority on real robots. However, regarding cross-task transfer, the current experiments support partial zero-shot transfer of already-accumulated knowledge, rather than stable and strong generalization to all unseen tasks; this is closely tied to how well the generated skills match unseen tasks, task similarity, and so on. We will continue to extend research on cross-task transfer in the future.

Q2

Thank you for pointing out this lack of clarity. In the original comparison, the ASPIRE baseline used Claude Opus 4.6, while META-Robot's main results used GPT-6 Astra. We agree that this comparison alone cannot separate the contribution of the framework itself from that of the language model.

To this end, in addition to the META-Robot / GPT-6 Astra versus ASPIRE / GPT-6 Astra comparison above, we also add results for META-Robot / Claude Opus 4.6 versus ASPIRE / Claude Opus 4.6, and we report the base performance of the various LLM models we tested on LIBERO-Pro, as shown in the table below:

MethodObj/TaskObj/PosGoal/TaskGoal/PosSpatial/TaskSpatial/Pos
META-Robot / Chat GPT-6 Astra99.00%99.60%69.80%85.80%95.80%91.40%
ASPIRE / Chat GPT-6 Astra98.20%99.80%67.20%85.00%81.20%79.60%
META-Robot / Claude Opus 4.696.40%98.20%65.60%84.60%78.80%76.80%
META-Robot (DeepSeek-V4-Flash)88.60%98.00%64.60%72.20%88.40%83.00%
ASPIRE / Claude Opus 4.695.00%98.00%45.00%81.00%60.00%51.00%
Gemini 3.1 Pro Preview22.00%20.40%18.00%26.00%10.00%9.00%
Claude Opus 4.620.00%9.00%16.00%38.00%2.00%15.00%
DeepSeek / V4-Flash16.00%5.00%0.00%12.80%0.00%10.00%
GPT-6 Astra88.60%61.00%42.00%76.60%43.80%27.60%

In summary, beyond the addition of predicates — which allow our framework to evaluate not only task execution but also the system itself, i.e., whether the system's judgments about task or skill execution are accurate and effective enough, which is what we mean by needing to extend not only capability but also cognition, and which gives us better performance on some widely used benchmarks — more importantly, our system can understand proprioception, which has enabled progress on tasks such as Weight comparison that other approaches cannot complete at all.

Finally, I really appreciate your comments and questions. We look forward to your reply and hope to work with you on something valuable for the community.

Reviewer_RFKCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

META-Robot is a code-as-policy agent that writes, together with each program, action-level predicates (post-conditions such as grasp verified) and checks them after every skill call. Failed predicates trigger local repair, then global repair and full-task re-execution; validated repairs and their predicates are stored in coupled skill and predicate libraries, and the agent can add proprioceptive signals (joint torque, gripper state) as verification evidence. It is evaluated on LIBERO-Pro, a predicate ablation, zero-shot transfer to LIBERO-Pro Long, one designed proprioception task, and seven real-robot tasks (8 trials each).

Critical strengths
  1. Per-step predicate checks give the repair loop a concrete failure signal; removing them lowers success in all six LIBERO-Pro settings (Table 2, e.g. Spatial–Task 95.8 → 76.0).
  2. Using torque and gripper feedback to verify contact and grasping (thin towel, thin sheet 50% → 78%, bottle weighing) addresses a real gap in vision-only code agents.
Critical weaknesses
  1. The Table 1 numbers are not comparable to the baselines as reported. The agent repairs and re-executes tasks in both settings (Sec. 4.1), and Sec. 4.4 stresses that only the transfer experiment is done "without additional debugging", so the main results appear to involve debugging on the scored tasks. The only debugging-free setting gives 0.0–40.2% (Task) and 0.0–21.2% (Swap), far below the 85–99% in Table 1. In addition, Spatial–Pos covers only 8 of 10 tasks (89.25 = 357/400), and the LLM is confounded with the method: with DeepSeek-V4-Flash, META-Robot trails ASPIRE on Object–Task (88.6 vs. 95.0) and Goal–Pos (72.2 vs. 81.0), and the baselines' LLMs are not stated.

  2. The main claim, that coupling predicates with skills is what enables capability scaling, is not tested. Table 2 removes predicates altogether, which only shows that checking post-conditions helps; checking post-conditions and repairing on failure is already common (e.g. ProgPrompt's assertions, Inner Monologue's success detection). There is no ablation that keeps the checks but drops the predicate library or the predicate–skill association, and no statistics on library size, reuse or wrongly admitted predicates to support "self cognition" or "emergent cognition".

Questions for authors
  1. Were the programs and libraries for Table 1 built or repaired on the same tasks and seeds that are scored? How many repair attempts does one reported success allow, and who runs the "review gate" in Appendix A.5?
  2. Can you report Spatial–Pos on all ten tasks, and at least one baseline (ASPIRE or CaP-X) run with the same LLM?
  3. Can you add an ablation that keeps per-step predicate checks but removes the predicate library or the predicate–skill association?
  4. Real robot: what are the trial counts and results without proprioception for towel folding and breakfast preparation?
10 comments
AuthorsOpenReview ↗

Response to the weakness 1

Thank you for highlighting the evaluation protocol, baseline comparability, and evidence for the proposed mechanisms. We clarify these points and provide additional controlled experiments point by point below. We apologize for the slightly delayed response, as we conducted additional real-robot experiments to supplement our evaluation.

W1

The programs and knowledge base in Table 1 are developed on the tasks being evaluated, but each task uses seeds 51-65 for development to accumulate knowledge, with the remaining seeds used for testing. Therefore, this setting measures performance on known tasks under unseen initialization conditions, not zero-shot generalization on unseen tasks. The zero-shot test in the main text (Section 4.4) is the zero-shot generalization test on unseen tasks. For the specific seed split, the freezing procedure, and the repair rights during the scoring stage, please see Q1. We have distinguished, in the main text, between debugging during the development stage and execution during the testing stage, so as to avoid the overly general description of "iterative repair."

The LIBERO-Pro Long experiment in §4.4 examines the transfer of accumulated knowledge to unseen long-horizon tasks, using a fixed predicate library and skill library with no additional debugging or library updates — that is, it is a zero-shot generalization test on unseen tasks. This experiment differs from Table 1 in task distribution, adaptation conditions, and purpose, so we believe it may be inappropriate to directly attribute the difference in their success rates to whether debugging was performed, nor can Table 1's high success rates substitute for evidence of cross-task generalization.

Thank you again for your comment. We agree that cross-model comparisons cannot attribute performance differences entirely to the framework. In the original table, the DeepSeek-V4-Flash version achieves 88.6% and 72.2% on Object–Task and Goal–Pos, respectively, lower than ASPIRE's 95.0% and 81.0%; therefore, we will qualify the advantage claims in the original text and not claim superiority over the baselines for all models and settings. To isolate the contributions of the framework and the model, the same-model comparison is presented in the table below. (The basic settings of all the following experiments are the same as those in the main text.)

MethodObj/TaskObj/PosGoal/TaskGoal/PosSpatial/TaskSpatial/Pos
META-Robot / Chat GPT-6 Astra99.00%99.60%69.80%85.80%95.80%91.40%
ASPIRE / Chat GPT-6 Astra98.20%99.80%67.20%85.00%81.20%79.60%
META-Robot / Claude Opus 4.696.40%98.20%65.60%84.60%78.80%76.80%
ASPIRE / Claude Opus 4.695.00%98.00%45.00%81.00%60.00%51.00%

In addition, to better reflect the capability brought by the framework, we tested — in advance and without additional harness or other designs, and with the same number of development and test rounds — the performance on LIBERO-Pro of the various LLMs associated with our method and the baselines.

ModelObj/TaskObj/PosGoal/TaskGoal/PosSpatial/TaskSpatial/Pos
Gemini 3.1 Pro Preview22.00%20.40%18.00%26.00%10.00%9.00%
Claude Opus 4.620.00%9.00%16.00%38.00%2.00%15.00%
DeepSeek V4-Flash16.00%5.00%0.00%12.80%0.00%10.00%
GPT-6 Astra88.60%61.00%42.00%76.60%43.80%27.60%

Cap-X and RATs use Gemini 3.1 Pro Preview; ASPIRE uses Claude Opus 4.6.

You also mentioned that the original Spatial–Pos result covers only eight tasks and cannot be directly treated as the complete ten-task result. We now report the complete ten-task evaluation across four independent runs, with 50 test seeds per task in each run. The mean success rate is 91.4% (1828/2000); the run-level results are given in Q2. For META-Robot / DeepSeek-V4-Flash, extending the Spatial-Pos evaluation from eight to all ten tasks leaves the success rate unchanged at 83.00%.

AuthorsOpenReview ↗

Response to the weakness 2

W2

Thank you for your comment. We agree that additional ablation experiments are needed to show that the persistent predicate library and the predicate–skill association each bring benefits — that is, we need to examine the contributions of the reusable predicate library and of its explicit association with skills. We add the relevant experiments below.

Also, as you noted, post-condition checking and failure feedback have existing research foundations, such as the ProgPrompt (paper) assertions and Inner Monologue success detection you mentioned. However, we do more than write recovery actions into the task program and use execution feedback to guide subsequent planning. Further, we make the verification criteria themselves objects that can be continuously updated. META-Robot not only uses existing predicates to check execution outcomes, but can also add or revise predicates and their evidence requirements based on execution evidence, and — after verification — store and reuse them together with the corresponding skills. Therefore, knowledge accumulation includes both the skills of "how to execute" and the verification criteria of "how to judge whether the execution is effective." This extension is also the "Cognition" scaling referred to in this paper: the system gradually expands the action states it can distinguish, the failure conditions it can recognize, and the range of evidence it uses to judge whether execution is effective.

Ablation experiment on the reusable predicate library and its explicit association with skills.

I appericate your question. Both ablation groups below retain the stepwise predicate-checking and failure-triggering mechanisms: the first group removes the persistent predicate library — the current task can still generate and execute checks, but no cross-task predicates are stored or retrieved; the second group keeps the skill library and the predicate library but removes the explicit predicate–skill association, retrieving the two kinds of entries independently. Except for the ablated mechanism, the model, development/test seeds, execution budget, and observation permissions remain the same.

SettingObject (Task/Pos)Goal (Task/Pos)Spatial (Task/Pos)
Full META-Robot96.4%/98.2%65.6%/84.6%78.8%/76.8%
Keep checking but remove persistent predicate library90.2%/91.2%45.0%/72.8%63.2%/59.6%
Keep both libraries but remove explicit association94.0%/95.4%59.8%/79.0%70.2%/68.2%

The ablation results show that both the persistent predicate library and the explicit predicate–skill association help improve task success rates. With the stepwise-checking and failure-triggering mechanisms retained, removing the persistent predicate library lowers success rates across all six metrics, with Goal-Task dropping by 20.6 percentage points, indicating that checks generated only from the current task cannot replace verification knowledge accumulated across tasks. After keeping both libraries but removing the explicit association, success rates decrease across all six metrics, with Spatial Task and Pos each dropping by 8.6 percentage points, indicating that independently retrieving skills and predicates is still insufficient to fully exploit their combined effect. These results support that the performance gain comes not only from execution-time checking, but also from the continuous accumulation of verification knowledge and the explicit association between skills and their applicable conditions and verification basis.

The rest part we will report in the next one

AuthorsOpenReview ↗

Response to the weakness 2- Continuing from the previous one

Additionally, we report the predicate-library size, predicate reuse, wrongly admitted predicates, number of predicate revisions, and staged capability changes as follows. All the statistics below are from META-Robot / Claude Opus 4.6.

MetricStatistical basisResult
Library size and growthAggregated according to the six capability categories in Appendix A.5, separately counting deduplicated skill instances, action-level predicates, and skill–predicate associations; distinguishing general patterns from task-specific instances, and recording increments by development stage. Reusable skills (6 categories): (1) localization and role binding; (2) grasping and holding verification; (3) proprioception and dynamic-predicate determination; (4) transport and failure recovery; (5) manipulation and release-condition verification; (6) long-horizon task closed loop and skill evolution. Task-specific: specific object appearance recognition, drawer geometry and open/close strategies, task-specific placement positions and action parameters.6 categories of reusable skills/predicates:(1) 11/50 (2) 6/54 (3) 5/46 (4) 5/29 (5) 4/27 (6) 3/18 Total: 34/224 Task-specific (total skills / total predicates): Obj/Task: 13/71 Obj/Pos: 13/67 Goal/Task: 11/68 Goal/Pos: 9/42 Spatial/Task: 18/96 Spatial/Pos: 18/91 Total: 82/435
Predicate reuseThe proportion of predicates reused across different evaluation seeds within the same task.Task: reused predicates / total predicates Obj/Task: 203/295 Obj/Pos: 192/291 Goal/Task: 130/292 Goal/Pos: 126/266 Spatial/Task: 218/320 Spatial/Pos: 210/315
Wrongly admitted predicates numbersThe number of wrongly admitted predicates (predicates previously wrongly accepted, found during testing).Task: wrongly admitted predicates Obj/Task: 5 Obj/Pos: 8 Goal/Task: 30 Goal/Pos: 31 Spatial/Task: 12 Spatial/Pos: 16

The statistics show that META-Robot accumulated 34 reusable skills and 224 action-level predicates covering six capability categories, as well as 82 task-specific skills and 435 task-specific predicates, forming a structure that combines general knowledge with task-adapted knowledge. In testing across different seeds, the predicate-reuse ratios for Object and Spatial reach 66.0%–68.8%, indicating that existing verification conditions can be called repeatedly across different initial states. In contrast, the reuse ratio for Goal is 44.5%–47.4%, and the numbers of previously wrongly admitted predicates found during testing are 30 and 31, respectively, higher than in other settings, suggesting that verifying the applicability of goal-related conditions remains a weak link. These results show that the value of the knowledge base lies not only in its size growth, but also in whether the accumulated skills and predicates can match the current task and provide reliable execution and verification evidence.

In addition, to verify whether our system exhibits staged capability changes — and, further, whether there is emergence of cognition and capability — we performed, in chronological order, masked testing on the skills and corresponding predicates for the six task categories.

Knowledge retainedObj/TaskObj/PosGoal/TaskGoal/PosSpatial/TaskSpatial/Pos
Using knowledge generated in the first 25%30.8%28.6%22.4%40.0%27.2%26.0%
Using knowledge generated in the first 50%45.6%40.2%33.8%46.2%41.8%39.2%
Using knowledge generated in the first 75%85.6%88.2%52.3%71.6%67.2%65.0%
Using knowledge generated in the first 100%96.4%98.2%65.6%84.6%78.8%76.8%

The masked experiment that retains skills and their corresponding predicates by generation time shows that, as the available knowledge increases from the first 25% to all of it, success rates improve continuously in every setting. In particular, from the 50% to the 75% stage, Object-Task and Object-Pos improve by 40.0 and 48.0 percentage points respectively, showing clear staged gains and a certain emergence phenomenon. This trend suggests that performance improvement depends not only on the amount of knowledge, but also on the gradual completion of key skills, verification conditions, and their combinations.

Thank you for pointing out these shortcomings and your advice really. We will further update the paper's description of library size, predicate reuse, and the introduction of different kinds of skills, and so on. We look forward to further improving this work

Reviewer_JFzVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a code-as-policy agentic harness based on separate skill and predicate collections. While most of the method remains unclear and vague in description, the framework includes a clear structure of how skills are evolved and ordered in a tree structure from general to specific, and how skills are selected based on pre- and post predicates. The framework is evaluated on the LIBERO-Pro simualtion benchmark and validated on some real-world robot tasks.

The paper presentation is full of AI artifacts and far from publication quality.

Critical strengths
  • A clear strength is the evaluation with DeepSeek in the simulation benchmark. It shows that the framework works with differnt coding models and signifies a reproducible result with an open-weights model.
  • The framework is validated on real-world robot trials, tough without any reference comparison of baselines.
Critical weaknesses
  • The experimental evidence is weak in many aspects and does not pass the bar for a scientifically conclusive experiment:
    • It is not specified if methods are finetuned with demonstration data on the evaluated tasks, and what their time and action budget for test-time-adaptation is per task.
    • For the agentic and code-as-policy baselines the underlying coding models are not specified, and there is no controlled comparison with the same agentic coding model under different frameworks
    • The improvement over ASPIRE is marginal except for spatial tasks, and there are no statistical bounds or standard deviations over seeds reported to indicate wheter any improvement is statistically significant
    • The comparison to prior art is overall restricted to a single benchmark. This is without reason, as there are many existing benchmarks to choose from, even with much higher rigour and real-word significance than LIBERO, e.g. RoboArena, Collosseum, ManipualtionNet
    • For the real world tests, success criteria are not specified and it is unclear whether success is evaluated by one of the authors, an independent judge, or a VLM
    • There is no refererence or comparison on the real-world tests, so we do not know how hard the tasks are. The overall number of trials is small.
    • None of the experiments reports execution time. Especially given the opportunity of the agent to retry and refine every skill, and the weak results with frozen skills in section 4.4, this is an important dimension to report and compare to baselines.
  • The description of the contributions and method is under-specific and the overall paper presentation lacks far behind the necessary standard for a publication, likely due to overuse of AI in writing. To provide some examples
    • The first contirbution claim is that the work "scales cognition and capability". The paper neither defines what it understands under cognition or capability, what scalability along such a dimension would mean, or how the experimental results validate this claimed contirbution.
    • The related work completely misses to relate the predicate framework to classical planning literature, where the concept of predicates for isolation of orchestrated subtasks originates, and does not cover any existing work that couple predictes with VLMs, e.g. https://doi.org/10.1109/LRA.2026.3726377
    • the paper is full of nonsense formualtions such as "The framework is grounded in the scaling of cognition" (line 150).
    • Section 3.3 as an example has no concrete information how proprioception is made available to skills. What are "physical semantics" (line 287)? How should a task "indicate" (line 289) what kind of signals it requires? Is this left to the user to specify? The claim that the work moves "beyond proprioception as a static input" is unfounded in literature. All tested prior works use streams of data instead of static inputs.
    • Line 314 claims tests over multiple code models (I am not sure what code models are?) and refers to Appendix A3, which contains no such information.
    • Line 458 refers to Figure 11, which does not exist in the paper.
    • The formatting of Section 4.5.1 is completely broken and indicates a lot of wasted whitespace.
Questions for authors

Most questions are already indicated in weaknesses, as the main overall weakness is a lack of information in the paper.

Here are some additional ones

  • Does Section 4.4 not indicate that most skills are completely overfit, since performans drops if you keep more than 50%?
  • In what way does the strongest baseline, ASPIRE, differ from the proposed framework? What can be concluded from the observed performance differences?
12 comments
AuthorsOpenReview ↗

Response to the weakness 1( 1.1& 1.2 )

Dear Reviewer, thank you for your questions and suggestions. We will answer your questions one by one and provide supplementary materials; we will incorporate the revisions into the main text and the appendix. We apologize for the slightly delayed response, as we conducted additional real-robot experiments to supplement our evaluation.

W1

1.1

Thank you for your question, but I am not sure what you mean by "finetuned with demonstration data on the evaluated tasks" — could you clarify? The framework we build is, strictly speaking, Code as Policy. Code-as-Policy (CaP) is a paradigm in which a language or vision-language model generates executable robot control programs, this definition comes from CAP-X(https://arxiv.org/abs/2603.22435). SFT/fine-tuning changes the parameters of a single model, whereas Code as Policy changes the interfaces and division of labor among multiple components, "gluing" them together through code into a closed-loop system that is executable, inspectable, and locally modifiable. Therefore Code as Policy (Our baselines: Aspire, Playful,CPA-X are all Code as Policy Methods) does not involve a parameterized training process.

We would appreciate further clarification from you. As stated in the paper, our experimental setup develops on seeds 51–65 and tests on seeds 1–50, which reflects cross-environment generalization; Section 4.4 is the cross-task generalization test, where no debugging was performed on the new task set.

Regarding the time and action budget for test-time adaptation that you mentioned, we report as follows. The reported time intervals describe the duration of one complete development + test session for a task across the four groups of experiments. The columns correspond to the six LIBERO-Pro task categories (Obj/Task, Obj/Pos, Goal/Task, Goal/Pos, Spatial/Task, Spatial/Pos).

MetricObj/PosObj/TaskGoal/PosGoal/TaskSpatial/PosSpatial/Task
Development + test time interval / minutes
META-Robot / Claude Opus 4.66–188–2410–2612–3412–3414–42
ASPIRE / Claude Opus 4.610–2610–3012–3414–3814–3816–50
Action budget / steps
META-Robot / Claude Opus 4.65500–63007700830094001130012400
ASPIRE / Claude Opus 4.65800–65008200880094001200012800

1.2

Thank you for your question and guidance. We will emphasize in the paper that our method uses DeepSeek V4-Flash, while the Cap-X baseline uses Gemini 3.1 Pro Preview, ASPIRE uses Claude Opus 4.6, and RATs uses Gemini 3.1 Pro Preview. Thank you again for the reminder; we will also state this in the main text.

As you suggested, after submission we also added the performance of META-Robot based on Claude Opus 4.6, compared as shown in the table below.

MethodObj/TaskObj/PosGoal/TaskGoal/PosSpatial/TaskSpatial/Pos
META-Robot / Claude Opus 4.696.40%98.20%65.60%84.60%78.80%76.80%
ASPIRE / Claude Opus 4.695.00%98.00%45.00%81.00%60.00%51.00%

As can be seen, with the same model, its performance is still below our framework. (Note: we treat 15 development seeds + 50 test seeds as one group of experiments, and what we compare here is the mean of our four repeated groups.)

I must appreciate again for your suggestions and guidance; we hope this is satisfactory.

AuthorsOpenReview ↗

Response to the weakness 1( 1.3& 1.4)

1.3

Thank you for the reminder; these statistics already existed in our experiment logs. We provide the standard deviation and statistical bounds across four independent runs here (50 seeds, 4 groups of development + testing in total). As for other baselines, due to time constraints, we reproduced each once and confirmed alignment with their reported results, so we did not run them multiple times; moreover, the relevant public reports do not provide complete standard-deviation and statistical-bound information. We will incorporate these revisions into the main text later.

META-Robot / GPT-6 AstraObj/TaskObj/PosGoal/TaskGoal/PosSpatial/TaskSpatial/Pos
Avg99.00%99.60%69.80%85.80%95.80%91.40%
Min98.00%99.40%67.40%83.00%95.00%90.20%
Max100.00%100.00%74.00%89.20%98.00%92.00%
Std (%)0.820.282.892.561.470.85

The Spatial-Pos results cover all ten tasks. The success rates across four independent runs are 91.4%, 90.2%, 92.0%, and 92.0%, corresponding to 457, 451, 460, and 460 successes out of 500 trials per run. The mean is 91.4% (1828/2000), and the sample standard deviation is 0.85 percentage points. For META-Robot / DeepSeek-V4-Flash, extending the Spatial-Pos evaluation from eight to all ten tasks leaves the success rate unchanged at 83.00%.

META-Robot / Claude Opus 4.6Obj/TaskObj/PosGoal/TaskGoal/PosSpatial/TaskSpatial/Pos
Avg96.40%98.20%65.60%84.60%78.80%76.80%
Min95.00%97.00%63.40%82.00%77.20%76.00%
Max98.00%99.00%68.00%86.80%80.80%78.00%
Std (%)1.230.911.971.981.570.91
META-Robot / DeepSeek-V4-FlashObj/TaskObj/PosGoal/TaskGoal/PosSpatial/TaskSpatial/Pos
Avg88.60%98.00%64.60%72.20%88.40%83.00%
Min87.60%97.40%62.60%70.00%87.60%82.40%
Max89.60%98.60%66.80%75.80%90.00%84.00%
Std (%)0.950.521.722.541.080.71

Additionally, we report the performance of different models by themselves on LIBERO-Pro, without any additional Harness or system guidance, for your reference. We find that DeepSeek V4-Flash's own capability is weaker than Claude Opus 4.6 (the model used by ASPIRE), as shown in the table below; yet META-Robot / Claude Opus 4.6 exceeds ASPIRE, so our advantage is actually not limited to spatial tasks.

ModelObj/TaskObj/PosGoal/TaskGoal/PosSpatial/TaskSpatial/Pos
Claude Opus 4.620.00%9.00%16.00%38.00%2.00%15.00%
DeepSeek V4-Flash16.00%5.00%0.00%12.80%0.00%10.00%
GPT-6 Astra88.60%61.00%42.00%76.60%43.80%27.60%

1.4

Thank you for the reminder. At the time, due to time constraints, we chose LIBERO-Pro (proposed in October 2025), which is the most commonly used benchmark in the robotic-agentic field for works such as Cap-X and Playful. In addition, to supplement long-horizon testing in advance, we later added results on RoboCasa365 (2026.3), as shown in the table below. Due to time constraints, however, we were unable to implement Cap-X and others on RoboCasa, so we compare against HarnessVLA (2026.7) using its publicly available code for reproduction. Since it is uncommon in the agentic-robotics community to test on the benchmarks you mentioned, we could not prepare in advance, nor reproduce the baselines on those datasets in time. We apologize again. We will strive to supplement these in the future; thank you very much for your suggestions and reminders.

For RoboCasa365, we use the target50 split, comprising 18 Atomic-Seen tasks, 16 Composite-Seen tasks, and 16 Composite-Unseen tasks. For each task, seed 0 is used for development and memory construction and is excluded from evaluation; seeds 1–5 are used for testing. We conduct four experimental runs, yielding 50 × 5 × 4 = 1,000 evaluation episodes per method. The corresponding evaluation denominators are 360 for Atomic-Seen and 320 each for Composite-Seen and Composite-Unseen. Each group-specific success rate is computed from the total number of successful episodes across the four runs divided by its corresponding evaluation denominator. META-Robot / GPT-6 Astra achieves success rates of 94.44% on Atomic-Seen and 86.25% on Composite-Seen.

MethodAtomic-SeenComposite-SeenComposite-Unseen
META-Robot / GPT-6 Astra94.44%86.25%85.00%
Harness VLA — Codex / GPT-6 Astra87.78%43.75%42.50%
Harness VLA — Claude Opus 4.780.00%47.50%15.00%
AuthorsOpenReview ↗

Response to the weakness 1( 1.5)

1.5

Thank you for your reminder and suggestions.

For META-Robot, the real-robot experiments follow the model configuration and experimental settings reported in the original manuscript. For the real-world tests, the success criteria are as follows: we use both human evaluation (one author responsible for the real-world test) and a VLM judge. After each experiment ends, the human makes judgments based on observations of the whole trial according to the detailed criterion. Meanwhile, we give a picture of the final status to the VLM to judge whether it succeeded.

For all 7 tasks, they must be done without any human intervention (no manual movement of objects, no resetting, no object delivery during the process).

Tennis Ball Pick-and-Place

  • It really grabs the ball: the gripper closes on the ball and lifts it off the table.
  • No drops: the ball never slips out and falls on the way. (If it drops, the trial fails even if it ends up in the right place.)
  • It gets to the goal: the ball ends up inside the target container.
  • It lets go: the gripper opens and moves away, so the ball is no longer held.
  • The ball stays put: after release, it falls into the box.
  • Nothing else gets knocked over: the box on the table moves no more than 1 cm.
  • The arm backs off: the arm leaves the workspace or returns to a neutral pose and doesn't block the target.

Put-General

  • Right object: the robot picks up the object the instruction asked for, or in the right order of different objects.
  • Fully inside: the whole object is put inside the container when viewed from above. No part hangs over the rim.
  • Lets go and backs out: the gripper opens fully and leaves the container; the gripper is no longer holding or propping up the object.
  • No damage: the container (box) doesn't tip over.

Towel Folding

  • Grabs the right spot: the robot takes the correct corner or edge of the towel (within 5 cm), not just anywhere in the middle.
  • First fold works: after folding, the two edges line up. Corners are off by no more than 5 cm, and the fold line is off from straight by no more than 10°.
  • Second fold works: the folded towel covers no more than 60% of its original area and still looks like a rectangle.
  • Lies flat: the folded towel rests flat on the table. No curled-up or floating parts, and nothing hangs over the table edge.
  • Stays folded: after the robot lets go, the towel stays folded for 2 seconds or more and the layers don't spring open (≤ 10°).

Breakfast Preparation

  • Action done: the right food is put where it belongs, in the right order.
  • No drops: the three food items never slip out and fall on the way.
  • No dropped items: nothing falls off the table or onto the floor.
  • Right layout: the final arrangement matches the target — items are in the right relative order and place, each within 5 cm.
  • Stays stable: after everything is let go, nothing moves or topples over.

Instruction-Guided Cup Stacking

  • Understands the instruction: the robot picks the right color, size, and order of cups — for example, if told "put the red cup on the blue one," it must not do it the other way around.
  • Stacking works: the top cup's base goes inside the rim of the cup below.
  • Right order: the stacking order matches the instruction. If the order is reversed, the trial fails even if the stack is stable.
  • No collapse: after release, the stack stays up.
  • Gripper fully out: the gripper opens and moves away from the stack, not still holding anything.

Plug insertion

  • Right socket: the plug goes toward the correct socket or port — not the wrong port, not plugged in backward.
  • Solid grip: the plug doesn't slip out of the gripper during insertion. (If it slips, the trial fails.)
  • Fully inserted: the plug goes in deep enough — the gap between the plug base and the socket face is 2 mm or less, or it reaches at least 90% of the required insertion depth.
  • Correct orientation, no damage: the plug isn't forced in the wrong way, and neither the plug nor the socket is visibly bent or broken.
  • Gentle enough: the peak contact force stays under the limit, and the whole experiment shows no hard banging.
  • Release works: after the gripper opens, the plug is still fully inserted.
  • Nothing else moves: the power strip and nearby cables stay put.

Weight comparison

  • All objects handled: the robot picks up or lifts each object off the table.
  • Gives an answer: the robot clearly states or signals which one is heavier.
  • Answer is correct: the answer matches the true answer — this is the one hard requirement.
  • No drops or damage: neither object falls or gets bent.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank all reviewers for their careful reading and constructive suggestions. We briefly summarize the main revisions, the core contributions, and our responses to the remaining questions.

1. Main Revisions and Additional Experiments

The revised PDF adds the following evaluations:

-The LIBERO-Pro Spatial–Pos results have been extended from the original 8 tasks to the full 10 tasks.

-In both simulation and real-robot environments, we add comparisons with ASPIRE using the same base model.

-We separately remove the persistent predicate library and the explicit skill–predicate association to test the contributions of the two mechanisms.

-We add a new real-robot transfer experiment from tennis-ball pick-and-place to cup stacking.

-We add evaluations on 50 tasks in RoboCasa365, and supplement the statistics from four runs on LIBERO-Pro.

The relevant results are shown in Tables 1, 4–8, and 15.

2. Core Contribution: Coordinated Refinement of Skills and Verification Conditions

META-Robot explicitly associates reusable skills with action-level predicates and jointly refines them during the development stage. Predicates describe a skill's applicable conditions, expected state changes, and verification basis, thereby supporting subsequent planning, skill selection, and execution checking; the visual, proprioceptive, and failure feedback produced by skill execution is in turn used to improve the skill implementation and predicate conditions, forming a Planning–Execution–Verification closed loop. During the test stage, these programs and knowledge structures are frozen.

Table 5 directly tests the role of this association: while retaining the skill library, the predicate library, stepwise checking, and failure handling, and removing only the explicit association between skills and predicates, the success rates in all six evaluation settings decrease. This provides ablation evidence for the practical contribution of the coupling between the two.

3. Responses to Remaining Questions

-Reviewer JFzV: Experimental protocol and comparability. Appendices B.12–B.15 specify the 15 development seeds and 50 non-overlapping test seeds for each task, as well as the frozen-test protocol, and separately report the development-seed re-evaluation and the held-out test results. The newly added LIBERO → LIBERO-Pro experiment restricts META-Robot's development scope to the original LIBERO tasks; its average success rate is 47.60%, higher than π0.5's 12.83%. This experiment controls the source-task scope but does not yet fully match the amount of environment interaction and the computational budget.

-Reviewer RFKC: Cross-task generalization and continual capability expansion. Section 4.4 and Tables 7 and 17 provide transfer evidence beyond reuse across different seeds of the same task. Transfer to previously unseen Long tasks achieves partial success, and the effect varies with the perturbation type and the retained knowledge. The current results support cross-task transfer and knowledge reuse within the evaluated scope; continual improvement over longer task sequences still requires further verification.

-Reviewer Hswa: Real-robot experiments and novelty. Table 6 adds the real-robot comparison with ASPIRE, and Table 13 reports the proprioceptive-feedback ablation. Fig. 3 illustrates how physical interaction feedback is converted into reusable skills and verification conditions, while Table 5 tests the contribution of the skill–predicate association. More complex and longer-horizon real-robot tasks remain an important direction for future validation.

Thank you for your valuable feedback and discussions. We look forward to refining this work together and hope our contributions can benefit this cutting-edge research community.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.