acceptodds
Under review as a conference paper at ICLR 2027

RoboFoundry: System-as-Policy Evolution for Self-Learning Embodied Agents

Abstract

A foundation model should not act in isolation as an embodied agent. Yet, existing methods often optimize individual components of the agent stack, such as memory, context, skills, or action interfaces, rather than treating the supporting system itself as a unified policy. Moreover, interaction alone does not yield self-improvement unless execution experience is converted into persistent, validated system changes. We therefore propose RoboFoundry, the first embodied agentic framework that formulates this process as Self-Evolving System-as-Policy. RoboFoundry diagnoses capability gaps in decision-making and memory management, converts execution traces into validated task-specific system updates, and promotes recurring improvements to the general system. Evolution operates over two complementary surfaces: a context system that manages active internal context and persistent file-system memory, and a hierarchical skill system that organizes atomic skills, reusable compositions, and failure-conditioned recovery. A shared semantic interface separates embodiment-invariant decisions from embodiment-specific execution, allowing evolved system capabilities to transfer across heterogeneous robots. On EmbodiedBench, RoboFoundry achieves state-of-the-art performance, notably improving GPT-5.5 by 27.8%. It also brings Qwen3.7-Plus to near parity with GPT-5.5 (70.3% vs. 72.7%), showing consistent gains from system-as-policy evolution across foundation models. For long-horizon memory, RoboFoundry outperforms all baselines on RoboMemArena by at least 39.0%, even against methods assisted by external foundation models. On LIBERO-PRO, it further outperforms Cap-Agent0 by 243.8%–679.7% across all perturbation types. In real-world deployments, RoboFoundry demonstrates zero-shot transfer and online evolution across robots and tasks, highlighting its potential for fully autonomous embodied agents.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied Agents, Self-Evolving Systems, System-as-Policy, Context and Skill Management

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4SWFOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes RoboFoundry, an embodied agent framework that formulates the system supporting a foundation model as the policy and makes this system the unit of evolution. RoboFoundry localizes system-level bottlenecks as capability gaps in execution traces, edits only the responsible surface, and promotes generalized edits that pass held-out verification to the general system. Across three embodied simulation benchmarks, RoboFoundry improves embodied decision-making across foundation models of varying capability and outperforms existing methods on long-horizon memory and on manipulation under changes in object positions and task specifications. In real-robot experiments on a bimanual platform and a Unitree G1 humanoid, RoboFoundry transfers evolved skills to unseen objects and continues to evolve after deployment, as in a nesting-doll task where edits made between trials raise success to 70% over ten trials against 30% for the π0.5\pi0.5 model alone.

Critical strengths

RoboFoundry offers a promising structure that turns LLM-written task logic into robot actions and confines each revision to the component responsible for a failure. Embedding it in a learning loop that promotes reusable rules across tasks makes it a plausible basis for larger systems spanning many tasks and robots.

Extensive real-world experiments and multi-axis analyses substantiate that the evolved system transfers across robots and keeps improving after deployment, with gains that hold across backbones.

Critical weaknesses

In the EmbodiedBench experiments, the variant that removes part of RoboFoundry keeps a fixed general system and applies task-level repairs alone, which still rely on capability diagnosis. As a result, it remains less clear how much capability diagnosis itself adds beyond revising the system directly from failure records. Environment interactions are matched across variants, although reporting the inference used for diagnosis and editing would make the comparison easier to interpret.

While task-level repairs are validated through execution, promotion to the general system relies on a model judgment on recorded traces from other tasks rather than on executing the proposed updates on held-out tasks. Although such a judgment is a reasonable low-cost proxy, the paper does not establish how reliably it predicts actual transfer performance or whether promoted updates introduce regressions on other tasks. This weakens the support for the claim that system updates are validated.

Questions for authors

How much model inference do the diagnosis and editing steps use compared with the baselines, and how well do the scores that decide which edits enter the general system predict actual execution outcomes?

The per-task tables show where RoboFoundry gains most, but what do the tasks with large gains have in common compared with those with small or negative gains?

How often are rules deleted or replaced over the course of evolution?

Do the authors view this framework as a long-term architecture even as foundation models evolve to internalize the abilities that its system components currently provide? For example, if a model could diagnose its own failures and revise its behavior on its own, which part of the method would become the main bottleneck, and would the save-retrieve-use memory operations and the six capability categories for diagnosing failures still be justified?

5 comments
AuthorsOpenReview ↗

Reply to Reviewer 4SWF - Part 1

W1, W2, and Q1.1: Capability Diagnosis (Attribution) and Promotion Judgment (Verifier)

We thank the reviewer for these suggestions. We have added experiments to directly evaluate capability attribution and the reliability of the promotion verifier, as shown in Table 2 (Counterfactual attribution and verifier validation on EmbodiedBench).

1. Capability Attribution

To evaluate whether capability diagnosis improves repair, we add a counterfactual attribution experiment across all four EmbodiedBench suites using GPT-6 Astra, Qwen3.7-Plus, and GLM5.3-Flash.

For every initially failed task, we compare RoboFoundry's diagnosed capability with two randomly selected alternative capabilities (CT1 and CT2). The alternative conditions intentionally direct editing toward different capabilities, while keeping the same failure trace, pre-edit system, editing budget, and execution budget. This isolates the effect of capability-guided targeting rather than additional environment interaction.

BackboneInitial FailuresRoboFoundryCT1CT2CT Avg.
GPT-6 Astra29967/299 (22.4%)24/299 (8.0%)19/299 (6.4%)7.2%
Qwen3.7-Plus438120/438 (27.4%)44/438 (10.0%)37/438 (8.4%)9.2%
GLM5.3-Flash42088/420 (21.0%)30/420 (7.1%)25/420 (6.0%)6.5%

The results show a clear difference: diagnosis-guided edits achieve approximately 3× higher repair success than edits targeting randomly selected alternative capabilities. This supports the importance of capability attribution and targeted editing rather than revising arbitrary components from failure traces.

2. Promotion Verifier

At the task level, RoboFoundry validates candidate repairs through actual environment re-execution. A repair is accepted only when it resolves the original failure.

For general-system promotion, the verifier uses recorded successful repairs from other tasks with the same diagnosed capability. For each held-out trace, the candidate general edit generates a new task-level repair, which is compared against the previously validated repair and its execution feedback. The system policy assigns a trace-based score rxr_x, and Eq. (3) averages the score differences to compute Δcap\Delta_{\mathrm{cap}}. A candidate is accepted for promotion when Δcap≥0\Delta_{\mathrm{cap}}\geq0. We have clarified the scoring criteria in Section 2.3: “This score is based on the recorded successful repair and…”.

To test whether this judgment predicts actual execution outcomes, we have added a verifier ablation using the successful task-level repairs from the attribution experiment above. For each repair, we generate a candidate general edit and evaluate it using Eq. (3). We then execute both accepted and rejected candidates on the corresponding held-out tasks, comparing their performance against the pre-promotion system.

BackboneCandidate EditsAcceptedPositive Transfer (Accepted)RejectedPositive Transfer (Rejected)Fisher p-value
GPT-6 Astra674335/43 (81.4%)246/24 (25.0%)1.33e-5
Qwen3.7-Plus1207458/74 (78.4%)4612/46 (26.1%)1.75e-8
GLM5.3-Flash885240/52 (76.9%)369/36 (25.0%)2.60e-6
Overall275169133/169 (78.7%)10627/106 (25.5%)1.51e-18

Statistical significance is assessed using a two-sided Fisher's test comparing positive-transfer rates between accepted and rejected candidates. Across the three backbones, candidates accepted by the verifier are approximately 3× more likely to achieve positive transfer in real held-out execution than rejected candidates (78.7% vs. 25.5%). This provides direct evidence that the promotion judgment is a useful proxy for actual transfer, rather than relying solely on model-generated scores.

AuthorsOpenReview ↗

Reply to Reviewer 4SWF - Part 2

We acknowledge that positive transfer does not guarantee the absence of regressions on previously successful tasks. The current promotion check does not explicitly test cross-capability retention. In Section 3.1, we have clarified this and also report the inference overhead of diagnosis and editing separately from the matched environment-interaction budget: “We further evaluate the promotion verifier by…”.

Together, these experiments support two key components of RoboFoundry: capability-guided attribution improves repair success, and the trace-based verifier predicts which general-system edits are more likely to transfer successfully.

Q1.2: Inference and Promotion Calls

For EmbodiedBench, all methods use model inference during task execution under the same environment interaction budget. Compared with the raw-backbone baselines, RoboFoundry additionally performs one outer-loop Revise call after each rollout for capability diagnosis and system editing.

For a complete sweep of 1128 scored tasks, this corresponds to 1128 additional model calls, or one extra call per task rather than per environment step. Each evolution call has a budget of 4096 input and 4096 output tokens, corresponding to a maximum budget of 9.24M additional tokens under this one-call-per-revision setting.

Q2: Characteristics of High- and Low-Gain Tasks

We thank the reviewer for this question. We observe a pattern in the per-task results. Tasks with large gains tend to benefit from reusable reasoning, planning, and recovery strategies. For example, EB-Habitat Spatial improves across all five backbones, with GPT-5.5 increasing from 32.0% to 100.0% and GLM5.3-Flash from 34.0% to 94.0%. As illustrated in Appendix A.1.1, RoboFoundry can turn failures into reusable rules, such as observation checkpoints and feedback-guided recovery, that benefit subsequent tasks.

In contrast, small or negative gains often appear in tasks requiring scene-specific visual grounding and precise action refinement. For example, EB-Manipulation Visual decreases for four of the five backbones, including GPT-5.5 (58.3% to 41.7%) and Qwen3.7-Plus (61.1% to 44.4%). Such tasks may benefit less from general system updates, which can also interfere with previously effective task-specific strategies.

In Section 3.1, we have added this analysis to clarify that RoboFoundry tends to benefit most from failures that can be turned into reusable system-level improvements, while scene-specific failures may show smaller gains or even regressions: “Larger gains tend to occur in tasks…”.

Q3: Rule Update During Evolution

We thank the reviewer for this question. We analyze rule changes over the full EmbodiedBench on GPT-6 Astra, Qwen3.7-Plus, and GLM5.3-Flash. For every promoted update, we record each rule-level operation as addition, replacement, or deletion. Each operation is counted separately.

BackboneRule OperationsAdditionReplacementDeletion
GPT-6 Astra142101 (71.1%)29 (20.4%)12 (8.5%)
Qwen3.7-Plus166116 (69.9%)35 (21.1%)15 (9.0%)
GLM5.3-Flash153109 (71.2%)31 (20.3%)13 (8.5%)
Overall461326 (70.7%)95 (20.6%)40 (8.7%)

Across all three backbones, 20.6% of rule operations replace existing rules, while 8.7% delete them. Together, 29.3% of operations modify or remove existing knowledge rather than simply adding new rules.

RoboFoundry updates existing rules when later execution feedback suggests a better correction, while deletion removes rules that are no longer useful or become inconsistent with new experience. These results show that RoboFoundry does not simply accumulate rules, but continuously revises its persistent general system as new execution experience arrives.

Q4: Long-Term Role of RoboFoundry

We thank the reviewer for this question. We view RoboFoundry as a system-level architecture, rather than a temporary solution for weak foundation models. Our results already support this: even with the stronger GPT-6 Astra backbone, RoboFoundry improves EmbodiedBench from 71.9% to 78.0%, showing that system-level evolution remains useful as foundation models become stronger. Similarly, Qwen3.7-Plus + RoboFoundry reaches 70.3%, close to GPT-5.5 + RoboFoundry at 72.7%, showing that system evolution benefits backbones of different strengths.

AuthorsOpenReview ↗

Reply to Reviewer 4SWF - Part 3

If future models can diagnose and repair failures internally, the main bottleneck will shift from diagnosis itself to which improvements should persist, how they generalize across tasks and embodiments, and how to prevent regressions from system updates. These challenges remain important even when the backbone becomes more capable.

The Save–Retrieve–Utilize operations represent the basic process of managing persistent experience, regardless of whether memory is external or integrated into the model. Similarly, the six capability categories provide an interpretable structure for locating and improving failures, but they are not intended to be fixed or exhaustive. They can be refined or replaced as models develop stronger internal capabilities.

Thus, the core idea of RoboFoundry remains the same: execution experience should lead to persistent, validated, and reusable system improvements, while the specific diagnosis and memory mechanisms can evolve with foundation models.

Reviewer_1TZPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The claim is that an embodied agent isn’t the model, it’s the whole system around it: what reaches the prompt, what memory survives, how a decision becomes motion. RoboFoundry freezes the model and edits that system with file operations instead. A failed rollout is attributed to one of six capabilities, an edit lands on the context or the skill surface, and if the rerun works it’s abstracted, replayed on held-out traces from other tasks, and promoted into a general system. A semantic binding layer separates decisions from robot-specific execution. Evaluated on EmbodiedBench (five backbones), RoboMemArena, LIBERO-PRO, and real robots.

Critical strengths
  1. The appendix shows actual diffs. The A.1.1 recovery rule mentions no cleanser, sink or sofa, so it really is task-agnostic. That makes the central claim checkable.
  2. The towel transfer. Evolve on a blue towel only, over 80% on unseen green and pink towels vs under 20% for π0.5. Small sample, but there’s no simulator state to read on a real robot, so it’s the part I believe most.
Critical weaknesses
  1. The LIBERO-PRO headline numbers appear to read simulator ground-truth poses while being grouped as not using them. Table 4 reserves † for privileged poses, but A.3.1 shows get_sim_object_pose(...), A.3.2 opens by reading current object poses, and A.3.3 says plainly that both the initial and revised systems use simulator poses. So I can’t tell what † adds. That undercuts the comparison against Harness VLA and ASPIRE, and removes the basis for “perception is still the bottleneck,” which is inferred purely from the †/non-† gap. Needs a rerun, not an ablation.

  2. The attribution-and-scoping mechanism is the real contribution and isn’t validated. H(M)H(M) picks the capability, writes the edit, and scores it, with task success as the only external signal — so a wrong attribution, a bad diff, and noise look identical. The missing control is an edit on the wrong surface that fails to repair the same failure. And edits clearly aren’t independent: on EB-Manipulation Visual, four of five backbones get worse after evolution (GLM5.3-Flash 72.2 → 47.2, Qwen3.7-Plus 61.1 → 44.4, GPT-5.5 58.3 → 41.7, Qwen3.8-27B 58.3 → 47.2), while EB-Habitat Spatial goes 32.0 → 100.0 for GPT-5.5. Nothing in the promotion check tests whether a candidate breaks a capability that already worked, and the paper never mentions this.

Questions for authors
  1. Figure 4 (GPT-5.5, full data) and Table 1 (same backbone) disagree in both directions and by large margins: ALFRED 92 vs 84.0, Habitat 74 vs 88.7, Navigation 48 vs 72.0, Manipulation 64 vs 45.9. The two can’t be measuring the same thing. Are the 50 held-out tasks per suite drawn to be representative, and are Table 1’s reported tasks disjoint from the tasks used for evolution? Please give held-out results for all five backbones.
  2. Does plain RoboFoundry call get_sim_object_pose? If so, how is it “without privileged poses”? State exactly what simulator state † and non-† each access.
  3. Did you check whether a promoted edit degrades a capability that already worked — specifically on EB-Manipulation Visual?
  4. How is rxr_x computed, and how well does the verifier’s score track real success? Have you tried misattributing a failure on purpose?
  5. What does evolution cost in tokens/API calls, versus spending the same budget on a stronger backbone?
  6. Minor: Tables 1 and 8 disagree (54.9 vs 54.8, 45.9 vs 46.1), and the Qwen3.7-Plus gain is printed as +10.4pp though 70.3 − 60.0 = 10.3.
7 comments
AuthorsOpenReview ↗

Reply to Reviewer 1TZP - Part 1

W1 and Q2: LIBERO-PRO Privileged Pose Setting

Thank you for pointing this out. We agree that Appendix A.3 did not clearly distinguish the privileged diagnostic examples from the non-privileged LIBERO-PRO setting, which made the source of object poses ambiguous. We have added clarifications in A.3 and A.3.1, “The evolution examples below use…” and “The following code uses privileged object-pose…”. We have also added a case with no privileged object pose in A.3: Non-privileged pose access.

The examples in A.3.1–A.3.3 were selected to illustrate how RoboFoundry changes execution abstractions during evolution. In particular, the pose-related example in A.3.3 comes from the privileged-pose diagnostic setting, rather than the non-† evaluation in Table 4. We have explicitly labeled these examples and added a non-privileged example showing how object poses are estimated from RGB-D observations instead of simulator state in A.3.

For the non-† results in Table 4, RoboFoundry does not access simulator ground-truth object poses. Following the perception interface design of CaP-X [1], object poses are obtained differently in the two settings:

  • RoboFoundry (non-†): RGB-D observations, camera calibration, and robot state → object grounding/segmentation → geometric pose estimation → estimated object pose. No simulator object position or orientation is exposed to the policy (same as FrankaLiberoApi.get_object_pose() in CaP-X).
  • RoboFoundry†: Object poses are obtained directly from simulator ground truth, as illustrated by the privileged helper get_sim_object_pose() in Appendix A.3 (same as FrankaLiberoPrivilegedApi.get_object_pose() in CaP-X).

The rest of the execution and evolution pipeline remains unchanged. Thus, the †/non-† performance gap reflects the effect of providing more accurate object poses, supporting our observation that perception remains an important limitation.

[1] Fu, L., Yu, J., El-Refai, K., Kou, E., Xue, H., Huang, H., ... & Fan, L. (2026). CaP-X: A framework for benchmarking and improving coding agents for robot manipulation. arXiv preprint arXiv:2603.2243

W2 and Q3: Attribution, Scoping, and Cross-Capability Regression

We thank the reviewer for pointing this out. We agree that the current results do not directly isolate the effect of attribution and scoping. As suggested by the reviewer, we add a counterfactual attribution experiment over the full EmbodiedBench using GPT-6 Astra, Qwen3.7-Plus, and GLM5.3-Flash to directly validate attribution and scoping. This tests whether the same failure is repaired more reliably when RoboFoundry edits the diagnosed capability surface than when the edit is forced onto a randomly selected alternative surface. For every failed task, we compare RoboFoundry's diagnosed capability with two randomly selected alternative capabilities, CT1 and CT2, under the same failure trace, pre-edit system, editing budget, and execution budget.

The results show a clear gap. RoboFoundry repairs 22.4% of the failed tasks for GPT-6 Astra, compared with 7.2% for the average counterfactual condition; 27.4% vs. 9.2% for Qwen3.7-Plus; and 21.0% vs. 6.5% for GLM5.3-Flash. Thus, diagnosis-guided edits achieve around 3× higher repair success than edits targeting randomly selected alternative capabilities.

On the other hand, EB-Manipulation Visual is an evaluation subset, rather than one of RoboFoundry's six diagnosed capabilities. A Visual task can involve perception, reasoning, planning, and recovery at the same time. Therefore, a decrease in Visual does not directly mean that one previously validated capability has been overwritten.

We also do not assume that general-system evolution improves every individual subset monotonically. RoboFoundry-Lite provides a direct control because it keeps task-level repair but removes persistent general-system evolution. For GPT-5.5, Visual decreases from 17/36 to 15/36 successful tasks from Lite to Full, while the overall EB-Manipulation success rate increases from 39.5% to 46.1%. For Qwen3.7-Plus, Visual remains unchanged at 16/36 successful tasks, while the overall success rate increases from 37.3% to 47.8%.

This reflects the difference between local specialization and general capability improvement. Scene-specific Visual corrections can be effective locally, while general-system evolution promotes reusable changes that improve performance across the broader task distribution. We therefore do not claim that every promoted update improves every fine-grained subset independently. We also acknowledge that the current promotion check does not explicitly test whether an update affects previously successful tasks from other capabilities.

AuthorsOpenReview ↗

Reply to Reviewer 1TZP - Part 2

Together, these results show that general-system evolution provides a clear gain across the broader task distribution, while the counterfactual experiment directly supports the importance of selecting the diagnosed capability surface rather than editing an arbitrary alternative surface.

Initial Performance on EmbodiedBench

BackboneALFREDHabitatNavigationManipulationTotal SuccessInitial Failures
GPT-6 Astra262/300 (87.3%)226/300 (75.3%)231/300 (77.0%)110/228 (48.2%)829/1128 (73.5%)299
Qwen3.7-Plus220/300 (73.3%)195/300 (65.0%)193/300 (64.3%)82/228 (36.0%)690/1128 (61.2%)438
GLM5.3-Flash215/300 (71.7%)182/300 (60.7%)213/300 (71.0%)98/228 (43.0%)708/1128 (62.8%)420

Counterfactual Attribution: GPT-6 Astra

ConditionALFREDHabitatNavigationManipulationTotal Repair
RoboFoundry8/38 (21.1%)34/74 (45.9%)10/69 (14.5%)15/118 (12.7%)67/299 (22.4%)
CT13/38 (7.9%)10/74 (13.5%)4/69 (5.8%)7/118 (5.9%)24/299 (8.0%)
CT22/38 (5.3%)8/74 (10.8%)3/69 (4.3%)6/118 (5.1%)19/299 (6.4%)

Counterfactual Attribution: Qwen3.7-Plus

ConditionALFREDHabitatNavigationManipulationTotal Repair
RoboFoundry24/80 (30.0%)46/105 (43.8%)23/107 (21.5%)27/146 (18.5%)120/438 (27.4%)
CT17/80 (8.8%)14/105 (13.3%)9/107 (8.4%)14/146 (9.6%)44/438 (10.0%)
CT25/80 (6.3%)12/105 (11.4%)8/107 (7.5%)12/146 (8.2%)37/438 (8.4%)

Counterfactual Attribution: GLM5.3-Flash

ConditionALFREDHabitatNavigationManipulationTotal Repair
RoboFoundry25/85 (29.4%)43/118 (36.4%)5/87 (5.7%)15/130 (11.5%)88/420 (21.0%)
CT18/85 (9.4%)13/118 (11.0%)3/87 (3.4%)6/130 (4.6%)30/420 (7.1%)
CT27/85 (8.2%)11/118 (9.3%)2/87 (2.3%)5/130 (3.8%)25/420 (6.0%)

Note: All results are reported as successful tasks / evaluated tasks (success rate). CT1 and CT2 denote two counterfactual conditions with randomly selected alternative capability targets. Repair success is evaluated on the same initially failed tasks for each backbone.

Q1: Table 1 (Full Online Evaluation) and Figure 4 (Held-Out Scaling)

Thank you for raising this point. Table 1 and Figure 4 use different evaluation protocols. Table 1 reports online evolution on the full EmbodiedBench, covering 1128 tasks across four suites. RoboFoundry starts evolving from the first scored episode, and episodes used for evolution are also included in the reported score. In contrast, Figure 4 is a separate held-out data-scaling experiment. We randomly sample 50 tasks from each suite as a fixed held-out set, leaving 928 held-in tasks in total. RoboFoundry evolves with 1/4, 1/2, or all of the held-in tasks and is then evaluated on the same fixed 200-task held-out set. Therefore, the Full point in Figure 4 measures the evolved system on unseen tasks, rather than the cumulative online performance reported in Table 1.

The held-out tasks are randomly sampled once and kept identical across all backbones and evolution-data settings, ensuring a fair comparison without manual task selection. These tasks are excluded from evolution and serve as a fixed diagnostic set for comparing different amounts of evolution data, rather than an estimate of full-suite performance. As requested, we also provide held-out results for the remaining four backbones, using the same fixed 50 held-out tasks per suite with 1/4, 1/2, and Full held-in evolution data. All four additional backbones show consistent improvement as the amount of evolution data increases, confirming that the scaling trend in Figure 4 is not specific to GPT-5.5.

We have added clarifications in Section 3: “To be noted, unlike Table 1, which reports online evolution over all…” and added the held-out experiments across all backbones in Table 3 (Held-out scaling on EmbodiedBench). We have clarified this sampling protocol in the revised paper and explicitly describe Figure 4 as a held-out scaling diagnostic.

AuthorsOpenReview ↗

Reply to Reviewer 1TZP - Part 3

GPT-6 Astra

Evolution DataALFREDHabitatNavigationManipulationAvg
1/442/50 (84%)39/50 (78%)36/50 (72%)24/50 (48%)141/200 (70.5%)
1/244/50 (88%)41/50 (82%)38/50 (76%)26/50 (52%)149/200 (74.5%)
Full46/50 (92%)43/50 (86%)39/50 (78%)28/50 (56%)156/200 (78.0%)

Qwen3.7-Plus

Evolution DataALFREDHabitatNavigationManipulationAvg
1/437/50 (74%)34/50 (68%)31/50 (62%)20/50 (40%)122/200 (61.0%)
1/239/50 (78%)37/50 (74%)34/50 (68%)22/50 (44%)132/200 (66.0%)
Full41/50 (82%)40/50 (80%)36/50 (72%)24/50 (48%)141/200 (70.5%)

GLM5.3-Flash

Evolution DataALFREDHabitatNavigationManipulationAvg
1/436/50 (72%)33/50 (66%)34/50 (68%)21/50 (42%)124/200 (62.0%)
1/238/50 (76%)35/50 (70%)35/50 (70%)23/50 (46%)131/200 (65.5%)
Full40/50 (80%)37/50 (74%)36/50 (72%)25/50 (50%)138/200 (69.0%)

Qwen3.8-27B

Evolution DataALFREDHabitatNavigationManipulationAvg
1/436/50 (72%)33/50 (66%)30/50 (60%)19/50 (38%)118/200 (59.0%)
1/238/50 (76%)36/50 (72%)32/50 (64%)21/50 (42%)127/200 (63.5%)
Full40/50 (80%)38/50 (76%)34/50 (68%)22/50 (44%)134/200 (67.0%)

Q4: Verifier and Attribution

Thank you for pointing this out. We add a clarification on the verifier score (rxr_x), including its scoring criteria. In Section 2.3, we add “This score is based on the recorded successful repair and…”. In Section 3.1, we add “We further validate the promotion verifier by…”, and Table2 (Counterfactual attribution and verifier validation on EmbodiedBench).

At the task level, RoboFoundry does not rely on the verifier alone: a candidate repair is accepted only after actual environment re-execution shows that the original failure is repaired. For general-system promotion, rxr_x is a trace-based verifier score produced by the system policy H(M)H(M). For each held-out trace with the same diagnosed capability, the candidate general rule produces a new task-level repair. The verifier compares this repair with the recorded repair that was already validated by successful execution, using its execution feedback as reference. Eq. (3) averages the score differences across held-out traces and promotes the general edit only when Δcap≥0\Delta_{\mathrm{cap}}\geq0.

To evaluate whether the verifier tracks real execution success, we add a verifier ablation using the same successful task-level repairs from W2 and Q3. For each successful repair, RoboFoundry abstracts a candidate general edit and evaluates it using Eq. (3). Candidates with Δcap≥0\Delta_{\mathrm{cap}}\geq0 are accepted for promotion, while the others are rejected. We then separately execute both accepted and rejected candidates on the corresponding held-out tasks and measure positive transfer over the pre-promotion system.

BackboneCandidate EditsAcceptedPositive Transfer (Accepted)RejectedPositive Transfer (Rejected)Fisher p-value
GPT-6 Astra674335/43 (81.4%)246/24 (25.0%)1.33e-5
Qwen3.7-Plus1207458/74 (78.4%)4612/46 (26.1%)1.75e-8
GLM5.3-Flash885240/52 (76.9%)369/36 (25.0%)2.60e-6
Overall275169133/169 (78.7%)10627/106 (25.5%)1.51e-18
Reviewer_g9VdOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces RoboFoundery, a framework to improve emboided agent by treating the system as policy for optimization. It presents the semantic interface to decouple embodiment-invarient decisions and specific executions. It further presents the policy optimization algorithm and ablated the performance under different settins.

Critical strengths

The authors presents a compelling conceptual design that optimize the whole system as a ploicy. They formulated with clear sementic separation of inner loop and outer loop to improve skills vs. system.

Critical weaknesses

The authors failed to present enough background information for general readers to understand the baseline methods. Furthermore, the ablations lack enough discussion to indicate the hypothesis behind the performance improvement.

Furthermore, the low performance of the baseline, e.g. openVLA has 0 success rate in LIBERO-PRO, indicates that the model might be right baseline.

Lastly, the authors could improve the presentation by providing more introduction on related work that benchmarked against later.

Questions for authors
  1. Task-specific corrections cannot guarantee performance on similar cases when the underlying general capability remains unchanged. It shows in EB-Manipulation experiment. Could the authors provide more insight for the failure case?
  2. Table 7 - why GPT-5.5 showed significant improvement on Visual using RoboFoundry-Lite?
3 comments
AuthorsOpenReview ↗

Reply to Reviewer g9Vd - Part 1

W1: Baseline Background, Related-Work Context, and Ablation Discussion

We thank the reviewer for these suggestions. These baselines are popular and plausible in the embodied AI community. We expand the background to better explain the differences between VLA policies (OpenVLA and π0.5\pi_{0.5}) and agent-based approaches (CaP-Agent0, Harness VLA, and ASPIRE), and clarify their relevance to RoboFoundry. We have revised this part as in Section 4, "VLA policies directly predict robot actions from…"

Regarding baseline performance, OpenVLA achieves 76.5% on standard LIBERO [1], but drops to 0% on LIBERO-PRO, consistent with the official benchmark results. This highlights the difficulty of handling distribution shifts, rather than an issue specific to our implementation. Importantly, we also compare against stronger agent-based methods. RoboFoundry outperforms Harness VLA (CC) (91.9% vs. 87.8%) in the non-privileged setting, showing that its improvement is not limited to comparisons with weaker VLA baselines.

Regarding the ablations, we aim to distinguish task-specific failure repair from persistent system-level evolution. RoboFoundry-Lite repairs individual failures (task-level), while RoboFoundry-Full retains and reuses these improvements (general-level). On EmbodiedBench with Qwen3.7-Plus, Full improves performance from 63.2% to 70.3%, supporting the benefit of persistent evolution beyond local repair. We have made these hypotheses and the role of each ablation clearer in Section 3.1: "Lite focuses on task-specific failure repair…".

[1] Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., ... & Finn, C. (2024). Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246.

Q1: Task-Specific Repair vs. General-System Evolution

We thank the reviewer for this observation. This limitation directly motivates the general-system evolution in RoboFoundry. Task-specific repair can resolve an individual failure without improving the underlying capability. As observed in EB-Manipulation, a correction that works for one object configuration or action sequence may not generalize to similar cases when the underlying reasoning or execution strategy remains unchanged.

A concrete example is provided in Appendix A.1.2, where the initial system fails to insert a red star into a shape sorter after 15 actions. The revised system adopts a grasp-preserving insertion strategy and succeeds in six actions. This illustrates how improving the underlying execution strategy can address a failure beyond simply changing an individual action sequence.

RoboFoundry addresses this by extending local corrections into reusable context and skills, allowing improvements to persist beyond individual tasks. The existing EB-Manipulation results (Table 8) further show the difference between task-level repair (Lite) and general-system evolution (Full).

BackboneVariantEB-Manipulation Avg.Complex InstructionSpatialVisual
GPT-5.5Lite39.5%31.3%39.6%47.2%
GPT-5.5Full46.1%43.8%41.7%41.7%
Qwen3.7-PlusLite37.3%31.3%37.5%44.4%
Qwen3.7-PlusFull47.8%52.1%43.8%44.4%

Full improves overall EB-Manipulation performance, particularly on Complex Instruction and Spatial tasks, although the gains are not uniform across all subsets. On the full EmbodiedBench, Full also improves performance from 63.2% to 70.3% with Qwen3.7-Plus, and from 65.5% to 72.7% with GPT-5.5. These results support the importance of general-system evolution beyond task-specific repair.

AuthorsOpenReview ↗

Reply to Reviewer g9Vd - Part 2

Q2: RoboFoundry-Lite on EmbodiedBench-Visual

We thank the reviewer for pointing this out. We believe the Visual result reflects the difference between local scene refinement and general capability improvement.

Visual failures can often be corrected effectively within the current scene, for example by refining object grounding, selecting a better target, or changing an action based on the current observation. RoboFoundry-Lite keeps these task-level corrections, so repeated refinement can strongly improve performance on this specific type of case. For GPT-5.5, this suggests that many Visual failures can be resolved through local refinement without changing its underlying general capabilities. This explains the significant improvement from 65.0% (baseline) to 80.0% (Lite, +15.0 pp), even slightly outperforming Full at 76.7%.

However, these scene-specific corrections do not necessarily improve reusable capabilities such as reasoning, planning, recovery, or long-horizon task progress. RoboFoundry-Full promotes corrections into the general system when they are useful beyond the current task. As a result, Full may not be best on every individual subset, but it provides stronger overall performance on EB-Navigation (72.0% vs. 69.7% overall, and 40.0% vs. 26.7% on Long Horizon for GPT-5.5).

We therefore do not interpret the Visual result as a failure of task-level refinement. Instead, it shows that local refinement and general-system evolution optimize different levels of performance: Lite can be very effective for a specific scene, while Full improves broader and more reusable capabilities.

AuthorsOpenReview ↗

Reply to Reviewer g9Vd

As the discussion period nears its end, we would like to ensure we have addressed your concerns. Your feedback and insights are valuable to us. Thank you for your time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.