acceptodds
Under review as a conference paper at ICLR 2027

SimEX: Simulation-Integrated Robotics AutoResearch

Abstract

Coding agents powered by large language models (LLMs) have shown remarkable abilities to autonomously reason about and achieve goals in the digital world. However, bringing this success to the physical world remains challenging. On the one hand, direct generation methods (e.g., Code as Policies) often suffer from the LLMs’ insufficient understanding of robots and physical environments. On the other hand, iterative trial-and-error tuning in the physical world (e.g., physical autoresearch) induces significant experimental cost and safety concerns. We introduce SimEX: Simulation-Integrated Robotics AutoResearch, an autoresearch framework that tightly integrates simulated experimentation, enabling coding agents to efficiently acquire physical capabilities for controlling real robots. SimEX operates in two stages. First, the agent conducts open-ended probe-and-optimize iterations in simulation, developing a robot toolbox with robust and generalizable capabilities. Second, the agent adapts the toolbox and the simulator together through only a few physical trials: each trial corrects the simulator, and the corrected simulator is used to diagnose failures and screen candidate repairs. We evaluate SimEX extensively in sim-to-sim settings and on physical robots. On challenging real-world manipulation tasks including towel folding, barcode scanning, and plate manipulation, SimEX enables coding agents to efficiently acquire robot skills without any demonstration and with only 10 minutes of real-robot interaction. These results suggest that simulation can be a critical component in achieving physical intelligence, not only as a source of training data for policies, but also as a laboratory where a coding agent iteratively develops capabilities through experimentation before acting on the robot.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Learning, LLM Agents, AutoResearch

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_gUs9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors provide a sim environment and an agentic toolbox to perform exploration for agentic robot coding, enabling search-based optimization in sim via agentic operations in Mujoco/FLASH (stage 1), and then physical/real adaptation procedure via real and sim experiments (stage 2). The agent is acting via calls to an API, adaptation during stage 2 are performed via
Stage 1 includes the LLM generating success scores and probing modifications of the policy along specific detected weakness phenomena. Stage 2 uses a real robot rollout by mimicking it in simulation, using simulation to generate improvement hypotheses, and then screen them in simulation before executing to test improvements on a real robot,
The authors test on a dual-YAM robot for three tasks, probing both sim-2-real and sim-2-sim, and checking the effect of LLM choice.

Critical strengths
  • The approach enables use of real->sim->real toward practical time agentic robotics.
  • Provides an important link between planning and
Critical weaknesses
  • Many details are missing to make the paper reproducible or detailed enough. Examples include details on the number of steps and kinds of modifications done inside the iterations, how weaknesses are represented, and are collected throughout the algorithmic search. See the "questions" sections for a set of aspects to improve on.
Questions for authors
  • The authors should probe different code-as-policies setups within sim-based planners, probe the influence within the current code-as-policies planner of modifications of the sim environment.
  • Would be good to check the sensitivity of the real-to-sim part to different mixture of real vs sim strategies, and sim variations as some approaches have been attempted, and the strategy for sim->real agentic adaptation is at the core of this paper.
  • Would be good to probe the weaknesses mentioned, as I assume you can get the LLM traces/code changes -- to show some insight into the CaP of stage 1.
  • Another ablation that would be relevant is the choice of weakness probing->focused weakness->dataset creation, vs versions that avoid focusing, or use a simpler strategy to favor difficult scenarios.
  • Simex - spell it out, is it exploration? Pls write as much.
  • Can you write also wall clock time, and spell out the time for all the call and sim steps? Especially as stating "10 minutes" in the abstract may be correct but is overselling the problem/solution.
  • Can you check the agent-judged success predicates, where do they succeed/fail?
  • Can you add missing papers and relate to them, such as “Graphs as policy”, “Revisiting the "Push-T" Robot Manipulation Task with Agentic Robotics”,”Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents”
Reviewer_H1rfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes an agentic system that enables coding agents to develop robot manipulation capabilities through simulation and adapt them to real robots with few-shot interactions. Specifically, it consists of two stages: 1) using self-generated tasks and evaluations in simulation to iteratively develop a reusable toolbox of perception and control capabilities, and 2) using a few real-robot trials to identify failures, improve the simulator, and test candidate code revisions before further physical execution. Evaluations are conducted across three manipulation task families in simulation and on a real dual-arm robot, showing improvements over existing baselines.

Critical strengths
  • The paper presents a compelling paradigm for developing robot capabilities primarily through simulation, followed by few-trial real-world adaptation. Particularly interesting is the idea that the simulator does not necessarily need to accurately reproduce the physical world, but can instead serve as an inexpensive environment for the agent to develop strategies, discover limitations, and iteratively improve its capabilities.
  • The real-robot demonstrations are compelling, covering diverse interactions and objects. The emergent strategies are also encouraging.
  • The paper includes extensive evaluations and ablations in both simulation and the real world.
Critical weaknesses
  • Unclear attribution of the capabilities to the proposed system versus the underlying APIs and harness: For agentic robotics systems, it is important to understand what capabilities are already provided through the robot APIs and execution environment, and what capabilities are actually developed by the agent. The paper specifies a common low-level interface across methods, including inverse kinematics, reachability queries, and motion control, which is helpful for controlled comparison. However, it remains difficult to understand the relative contributions of these existing capabilities and the newly developed toolbox. More concrete examples/experiments on the available/unavailable APIs and their corresponding effects on task success would help clarify the attribution of the performnace gains.
  • Insufficient understanding of the role and limitations of imperfect simulation: One particularly interesting aspect of the proposed method is that the simulator does not need to be an accurate digital twin. However, the paper does not systematically investigate how the accuracy of the simulator affects the performance of the overall system. While the authors provide ablations of the two stages and their components, it remains unclear how much of the improvement comes specifically from correcting the simulator based on physical feedback, as opposed to simply testing more candidate repairs in simulation. Evaluating different levels of simulation fidelity and isolating the effect of simulator correction would help establish this central claim.
  • Limited discussion of practical limitations: The limitations discussed in the paper focus primarily on the use of a single robot platform and manual scene resets. While valid, these do not fully capture the limitations specific to the proposed system. For example, the approach relies on possibly prepared simulation environment (or construction primitives), available object assets, and substantial agent-driven experimentation and their associated costs. Although the physical interaction budget is small, the simulation and coding-agent computation costs are considerable. It is also unclear how well the approach would scale to more diverse environments where constructing or correcting even an approximate simulator becomes difficult. A more targeted discussion of these trade-offs and potential failure cases would strengthen the paper.
Questions for authors

See weaknesses section.

Reviewer_TRKxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how an LLM coding agent can acquire manipulation skills without demonstrations and with very few physical trials. SimEX has the agent reconstruct the workspace in a simulator, then optimize a persistent "toolbox" that a one-shot code-as-policy call uses to write task programs. In Stage 1, the agent runs open-ended probe–focus–optimize cycles in simulation, generating its own tasks and success predicates, with capability-bank replay and a population of four agents. In Stage 2, the toolbox is adapted to one target task using five real trials. After each rollout, the agent replays it in simulation, edits the simulator until the replay matches, and screens candidate repairs in simulation. The paper also evaluates sim-to-sim: a MuJoCo/FLASH sandbox paired with Isaac Sim/MJWarp as the evaluation environment, over 21 tasks in three families. There SimEX averages 94/67/85% success, while no external baseline exceeds 35/38/18%. On a physical YAM robot it succeeds in 26/30 trials, versus at most 3/30 for any baseline.

Critical strengths
  1. The experimental results are promising and valid. SimEX matches or exceeds the best external baseline on all 21 tasks, and scores 85% vs. 18% on barcode. Results also show that both stages matter, and they interact. On barcode, Stage 1 alone scores 1%, SimEX without discovery 24%, SimEX with direct repair 58%, and full SimEX 85%. Figure 4 shows sandbox-guided repair ahead of direct repair at every adaptation rollout.

  2. The protocol is controlled and replicable. All methods share the API, initial toolbox, prompt, agent and population/iteration budgets. Results replicate across four coding agents, and cost is reported.

Critical weaknesses
  1. Simulator correction is not isolated. The abstract states "each trial corrects the simulator," but alignment is purely the agent's qualitative judgment , and no fidelity metric is reported. The only variant without alignment, direct repair, also removes hypothesis generation and screening.

  2. The Stage 1 → Stage 2 handoff is unspecified. In §2.3 and Table 2, four independent population members each produce their own toolbox, but the paper never says how one is selected or how they are merged. If the selection uses anything correlated with the evaluation tasks, that is leakage. It is also unclear whether Stage 1 is rerun for each of the five seeds. If it is not, the CIs omit the variance of the most expensive stage.

Questions for authors
  1. How is the Stage-1 output selected or merged from the four population members, and on what data?

  2. Are initial states randomized across the evaluation rollouts? How does a toolbox adapted on one task perform on the other six in its family?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.