acceptodds
Under review as a conference paper at ICLR 2027

EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot Planning

Abstract

A robot should be able to learn through experiments how unfamiliar objects behave and interact, then plan with that knowledge. It need not start from scratch: physics engines supply knowledge of motion and contact, but can omit entire mechanisms, such as glue curing, water heating, or wind. We present EMPIRIC, an agent that learns a residual world model: a physics engine extended with code for the missing mechanisms. The learned programs can introduce new forces, constraints, and hidden state, and Bayesian inference estimates their parameters and states from noisy observations. The resulting model lets the agent predict the outcomes of actions, choose informative experiments, and revise its hypotheses when predictions fail. Across five simulated domains, EMPIRIC learns interpretable, reusable models, and solves more tasks with fewer environment interactions than all three baselines. On a physical robot, it learns wind forces and domino masses to solve a manipulation task.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
code world models for planning, neuro-symbolic ai, robot learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_M7TyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents EMPIRIC, an agentic framework for learning residual world models through interaction for novel tasks that require trial and error. Starting from a supplied physics simulator, an LLM-based coding agent generates executable programs to model missing physical mechanisms. The framework further integrates Bayesian parameter inference, simulation-based planning, and informative experimentation to support robot manipulation under uncertainty.

The method is evaluated across five simulated manipulation domains, achieving 25/25 successful runs and outperforming three agent-based baselines in success rate and interaction efficiency. A real-robot experiment involving wind forces and domino manipulation further demonstrates its practical applicability.

Critical strengths
  1. Effective integration of physical priors with agentic world-model learning. Extending an existing physics simulator with executable missing mechanisms is a practically meaningful design choice. It allows the agent to leverage established rigid-body dynamics while modeling additional phenomena, including history-dependent effects. The resulting models are interpretable and can support planning across different task configurations.

  2. Strong empirical performance across diverse physical mechanisms. EMPIRIC achieves consistently high success rates across five simulated domains, substantially outperforming the evaluated agentic baselines. The real-robot demonstration also provides encouraging evidence that the framework can identify physical properties through interaction and use the resulting model for planning.

Critical weaknesses
  1. Limited methodological novelty beyond existing programmatic world-model learning and agentic frameworks. The central workflow of EMPIRIC is to use an LLM to generate executable world models (simulators), evaluating predictions against observations, revising the models, and planning with them. This has substantial overlap with existing approaches such as WorldCoder, TheoryCoder, and POMDP Coder. Similarly, residual physics modeling, Bayesian system identification, and information-driven experimentation are well-established techniques.

The principal distinction appears to be the use of a supplied physics engine as an inductive prior, with an LLM generating executable extensions for missing physical mechanisms. While this is a sensible and useful design, the paper does not clearly establish a fundamentally new learning algorithm or sufficiently explain what methodological capabilities go beyond combining existing techniques in an agentic structure.

Additionally, the high-level decisions regarding mechanism generation, model revision, and experiment selection remain largely delegated to the general-purpose coding agent, while parameter fitting and simulation-based planning rely on established techniques. Consequently, I find the contribution more convincing as a system integration effort than as a distinct methodological advance.

Furthermore, although the framework is evaluated across different physical mechanisms, it is unclear how much domain-specific engineering is needed to support a new environment, given the domain-dependent prompts and additional reference files described in Appendix D. This raises the question of whether the framework's generality comes from a distinct learning methodology or primarily from the flexibility of the coding agent and supplied infrastructure.

  1. Insufficiently controlled baseline comparisons to isolate the claimed contributions. The experimental comparisons conflate the proposed residual modeling approach with substantial differences in the infrastructure available to each agent. EMPIRIC receives a supplied executable physics simulator integrated with the robot's skill controllers, along with dedicated tools for parameter fitting, model validation, trajectory rehearsal, and plan refinement (Appendix B.1 and D.3).

In contrast, the Direct agent has no supplied simulator, while Standalone Simulator must construct its predictive model without the supplied base simulator. Although Direct + Scene receives scene assets and engine code, it is unclear whether it has access to comparable skill-level simulation, model-fitting, and planning interfaces (Section 4.1).

Appendix D.3 indicates that EMPIRIC can simulate robot skill execution before learning any missing mechanisms, including checking grasp feasibility and collisions. I believe this could already provide a substantial planning advantage. The authors also acknowledge in Section 4.1 that some baseline failures arise from incorrectly modeling robot skill execution, suggesting that the supplied simulation infrastructure may contribute significantly to the observed improvements.

While these comparisons are meaningful at the system level, they do not isolate the benefits of the proposed residual-model learning methodology. An important missing baseline is a coding agent equipped with the same base simulator, skill simulation, fitting/evaluation tools, and computational resources, but without EMPIRIC's prescribed residual-model representation and learning workflow.

Overall, the results demonstrate the effectiveness of the complete framework, but currently provide insufficient evidence that its proposed methodological contributions are responsible for the gains.

Questions for authors
  1. Could you compare EMPIRIC against the coding agent with access to the same base simulator, robot skill simulation, parameter-fitting/evaluation utilities, and computational budget, but without the proposed residual-model representation and prescribed learning workflow? This would help isolate the benefits of the proposed method from those of the supplied simulation infrastructure.

  2. Could you clarify the precise tool access and system prompts provided to Direct + Scene and Standalone Simulator, particularly whether they could rehearse robot skills using the same underlying physics engine and access equivalent trajectory-evaluation functions? This information is important for assessing the comparability of the experimental settings.

  3. Appendix D.3 reproduces the system prompt for only one Bridge run and notes that prompts for other domains contain domain-specific differences. The agent also receives reference files that are not reproduced in the paper. Could you clarify how much domain-specific engineering or prompt customization is required when applying EMPIRIC to a new environment with unfamiliar physical mechanisms?

While the test tasks evaluate new configurations within the same domains, it remains unclear how readily the framework can be extended to entirely new physical mechanisms without additional domain-specific engineering. To what extent does the proposed framework generalize to entirely new mechanisms without additional domain-specific modeling support?

3 comments
AuthorsOpenReview ↗

Response to Reviewer M7Ty (1/3): W1

We thank the reviewer for the careful reading and the constructive suggestions. We have revised the agent and added a baseline and an ablation for the comparison that the reviewer asks for (see our general response). All simulated results below are for the revised version.

W1.

Limited methodological novelty beyond existing programmatic world-model learning and agentic frameworks. The central workflow of EMPIRIC is to use an LLM to generate executable world models (simulators), evaluating predictions against observations, revising the models, and planning with them. This has substantial overlap with existing approaches such as WorldCoder, TheoryCoder, and POMDP Coder. Similarly, residual physics modeling, Bayesian system identification, and information-driven experimentation are well-established techniques.

We build on these techniques, but none of them lets a robot add a missing mechanism to a physics engine from its own experiments and then plan under uncertainty about it. Prior program world-model learners (WorldCoder, TheoryCoder, POMDP Coder) target grid worlds, games and discrete domains. Residual physics methods (SAIN, NeuralSim) fit corrections to data collected beforehand, and active identification (ASID) fits the parameters of a fixed model.

The principal distinction appears to be the use of a supplied physics engine as an inductive prior, with an LLM generating executable extensions for missing physical mechanisms. While this is a sensible and useful design, the paper does not clearly establish a fundamentally new learning algorithm or sufficiently explain what methodological capabilities go beyond combining existing techniques in an agentic structure.

Our contribution is a new formulation of what the robot learns: a program that extends a domain-agnostic base simulator (a physics engine with a model of the robot) with the missing mechanisms, their parameters and their hidden state, together with a belief over the parameters. The formulation gives two capabilities that the existing techniques lack: the robot learns both the parameters and the structure of its model, and one belief serves both learning and planning (the robot designs experiments to narrow it and rehearses its plans under it). Agents with only parts of this design solve 30 to 41 of 50 runs, and EMPIRIC solves 47.

Additionally, the high-level decisions regarding mechanism generation, model revision, and experiment selection remain largely delegated to the general-purpose coding agent, while parameter fitting and simulation-based planning rely on established techniques. Consequently, I find the contribution more convincing as a system integration effort than as a distinct methodological advance.

The same coding agent solves more runs with our model than without it: the four baselines use the same coding agent, skills and budget, and they solve 35 to 41 of 50 runs, against 47 for EMPIRIC (Table 1). The coding agent makes the high-level decisions by design, so that a stronger coding agent improves the system with no change to the harness. Earlier systems such as WorldCoder and ExoPredicator fix the learning flow in advance, which we see as a limitation and an inefficiency.

Furthermore, although the framework is evaluated across different physical mechanisms, it is unclear how much domain-specific engineering is needed to support a new environment, given the domain-dependent prompts and additional reference files described in Appendix D. This raises the question of whether the framework's generality comes from a distinct learning methodology or primarily from the flexibility of the coding agent and supplied infrastructure.

The revised agent needs no domain-specific engineering. The user provides the task definition, the scene assets, the sensor noise levels and the robot skills that the domain uses, which come from a shared library. The base simulator, system prompt, tools and reference files are the same in every domain. Appendix D reproduces the prompt and marks the three places that differ: the noise levels and the numbers of bodies and asset files.

AuthorsOpenReview ↗

Response to Reviewer M7Ty (2/3): W2

W2.

Insufficiently controlled baseline comparisons to isolate the claimed contributions. The experimental comparisons conflate the proposed residual modeling approach with substantial differences in the infrastructure available to each agent. EMPIRIC receives a supplied executable physics simulator integrated with the robot's skill controllers, along with dedicated tools for parameter fitting, model validation, trajectory rehearsal, and plan refinement (Appendix B.1 and D.3).

In contrast, the Direct agent has no supplied simulator, while Standalone Simulator must construct its predictive model without the supplied base simulator. Although Direct + Scene receives scene assets and engine code, it is unclear whether it has access to comparable skill-level simulation, model-fitting, and planning interfaces (Section 4.1).

The revised comparison separates the residual model from the supplied infrastructure, as the reviewer asks. All agents share the coding agent (Claude Opus 5), the skill library and the step budgets. The table below (Table 1 and Appendix A.4 in the paper) lists what else each agent receives.

AgentScene assetsBase simulator (engine, robot model)Mechanism learning interfaceHarness fitExplicit uncertaintySolved (of 50)
Direct agentnonononono35
Direct + sceneyesnononono41
WorldCoder (adapted)yesnononono38
Direct + base simulatoryesyesnonono37
No learningyesyesnono–30
No harness fittingyesyesyesnoyes41
No explicit uncertaintyyesyesyesyesno39
EMPIRICyesyesyesyesyes47

Direct + scene and WorldCoder (adapted), which the submission called Standalone sim., receive the scene assets (URDF and meshes, with no masses or frictions), and PyBullet is importable in their sandbox. They do not receive the base simulator, which holds the simulated robot, so they can rehearse a skill only in a simulator that they write themselves. All baselines can apply the task evaluator to the trajectories that they predict, as EMPIRIC can (Appendix A.4 lists each agent's tools).

Appendix D.3 indicates that EMPIRIC can simulate robot skill execution before learning any missing mechanisms, including checking grasp feasibility and collisions. I believe this could already provide a substantial planning advantage. The authors also acknowledge in Section 4.1 that some baseline failures arise from incorrectly modeling robot skill execution, suggesting that the supplied simulation infrastructure may contribute significantly to the observed improvements.

The base simulator's physics engine and robot model do not explain the gain by themselves. Direct + base simulator and No learning have both but cannot add mechanisms (see the table above), and they solve 37 and 30 of 50 runs, fewer than Direct + scene (41), which has no supplied simulator. In Domino, for example, the base simulator misses the spin of a domino struck at an angle, so No learning solves 1 of 10 runs. Some baseline failures do come from a wrong model of the robot's motion (Section 4.1 gives one in Fan), but the revised paper no longer attributes EMPIRIC's advantage to the robot model.

While these comparisons are meaningful at the system level, they do not isolate the benefits of the proposed residual-model learning methodology. An important missing baseline is a coding agent equipped with the same base simulator, skill simulation, fitting/evaluation tools, and computational resources, but without EMPIRIC's prescribed residual-model representation and learning workflow.

A coding agent with all of these components is basically EMPIRIC itself: the residual model is a subclass of the base simulator in which the agent writes its mechanisms, and the agent decides when to experiment, revise the model, fit and act. We therefore remove parts of this design. Direct + base simulator keeps the physics engine, the robot model and the task evaluator, but it cannot add mechanisms and has no fitting or rehearsal tools; it solves 37 of 50 runs, and EMPIRIC solves 47.

Overall, the results demonstrate the effectiveness of the complete framework, but currently provide insufficient evidence that its proposed methodological contributions are responsible for the gains.

Each part matters for a reliable agent, as three ablations now show: the agent loses 17 runs without the learned mechanisms, and 6 or 8 runs without the harness fit or the explicit uncertainty.

  • No learning keeps the base simulator and the rehearsal and planning tools, but it cannot add mechanisms or fit parameters. It solves 30 of 50 runs.
  • No harness fitting and No explicit uncertainty keep program learning and solve 41 and 39 of 50 runs. In Domino, where a single push that cannot be undone decides the outcome, they solve 5 of 10 runs each, against 10 of 10 for EMPIRIC.
AuthorsOpenReview ↗

Response to Reviewer M7Ty (3/3): Q1-Q4

Q1.

Could you compare EMPIRIC against the coding agent with access to the same base simulator, robot skill simulation, parameter-fitting/evaluation utilities, and computational budget, but without the proposed residual-model representation and prescribed learning workflow? This would help isolate the benefits of the proposed method from those of the supplied simulation infrastructure.

Please see our reply to W2: such an agent is basically EMPIRIC itself, so we compare against agents that each lack one part of it (37, 30, 41 and 39 of 50 runs, against 47 for EMPIRIC). All agents have the same budget of environment steps. EMPIRIC uses more computation within it, a median of 278 minutes per run against 50 for Direct + base simulator, and Section 6 and Appendix A.3 state this cost.

Q2.

Could you clarify the precise tool access and system prompts provided to Direct + Scene and Standalone Simulator, particularly whether they could rehearse robot skills using the same underlying physics engine and access equivalent trajectory-evaluation functions? This information is important for assessing the comparability of the experimental settings.

Both agents can use the same physics engine and the same task evaluator, but they do not receive the base simulator, which holds the simulated robot. Our reply to W2 gives the details, and the table there (Table 1 in the paper) lists what each agent receives.

Q3.

Appendix D.3 reproduces the system prompt for only one Bridge run and notes that prompts for other domains contain domain-specific differences. The agent also receives reference files that are not reproduced in the paper. Could you clarify how much domain-specific engineering or prompt customization is required when applying EMPIRIC to a new environment with unfamiliar physical mechanisms?

The revised version needs no prompt customization and no modeling code for a new environment (see our reply to W1).

Q4.

While the test tasks evaluate new configurations within the same domains, it remains unclear how readily the framework can be extended to entirely new physical mechanisms without additional domain-specific engineering. To what extent does the proposed framework generalize to entirely new mechanisms without additional domain-specific modeling support?

The mechanisms in our five domains are already new to the base simulator, which has no glue, lift, airflow, water or heat, and the agent adds each of them from its own experiments with no modeling code from the user. We did not test a domain outside these five. The main assumption is the rigid-body physics engine: for phenomena such as deformable objects, the base simulator can use an engine with soft-body support, or the agent's program can simulate the object itself, and we have not evaluated these extensions (Section 6).

Reviewer_eoSdOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

EMPIRIC is a framework that maintains a world simulator as code, updating its parameters and beliefs from new observations, and adding new mechanisms to the simulation when necessary. It uses the world model for planning, either to achieve its objective, or to explore and gather data to update its hypothesis and revise the model. In five simulated, EMPIRIC solves 25/25 runs.

Critical strengths
  • EMPIRIC takes advantage of both a physics simulator that models environment dynamics robustly and computationally efficiently, and a learnt world model that can adapt to the environment dynamics, update its parameters, and add missing mechanisms to improve the modelling capabilities.
Critical weaknesses
  • While the authors claim that the formulation can be extended to probabilistic dynamics, in practice they only learn a deterministic simulator. It is not apparent why they did not perform the probabilistic dynamics ablation if their formulation allows them to do so.
  • “Code as world models” idea is not new, and there are many related work in this, such as those mentioned in the paper’s related work section. However, the authors do not demonstrate any comparisons against these baselines apart from ablation studies that are “inspired by” other related work. The results graph is too coarse (only based on five runs) to be able to be considered as reported numbers for any future work to use as comparison points. The benchmark also seems too easy as a task since EMPIRIC saturates all of them.
  • While the authors claim the update mechanism for the code world model to be Bayesian-motivated, in practice only the parameters are fitted Bayesian-style, and the world model proposal and update itself does not take advantage of Bayesian nor gradient descent.
  • EMPIRIC’s ability to “plan” is not demonstrated – with full observation of the geometry in the scene, it is unclear whether EMPIRIC needs to plan and revise its plan at all, since a greedy method may be able to solve it.
  • Both the world model and the evaluation benchmarks are implemented in PyBullet, so there isn’t any evidence that the world model can learn to generalize beyond its prior.
  • The agent is given scene geometry as observations, which makes the task trivial, bypassing the need to use the image observations, and undermines the “learnt world model” claim.
  • The entire paper is clearly heavily written by AI with insufficient human revisions to make it more readable.
Questions for authors
  • L118: Doesn’t the environment evaluate the reward R evaluated on the ground truth state, rather than the noise-free observations? If the observation contains the target then the environment setup is limited to tasks with immediate rewards (such as object finding) and not tasks that involves long-term dependencies with delayed rewards.
  • L179: If the quantities such as “each glued join’s cure progress” is not kept track of by the engine, how does it know to store these properties in the recurrent state xresx_\text{res}?
  • Can you add more challenging benchmarks with a more fine-grained success metric that tests the planning capabilities of EMPIRIC?
  • Can you benchmark the performance of EMPIRIC against other existing work (e.g. those listed under “code world models” in the literature review) on benchmarks that are not based on PyBullet?
  • What is the decision criteria for EMPIRIC to add a new mechanism, as opposed to tuning the existing parameters to fit the new observation?
  • While the information gain measure Ij(ω)\mathcal{I}_j(\mathbb{\omega}) is mathematically interesting, this quantity is only used to rank experiments, and the LLM agent has the final call, so it is unclear whether this quantity is the optimal choice or simply an intuitively sound choice, and it is also unclear if the LLM agent requires this ranking at all. Could you conduct some ablation studies to verify whether other choices of ranking (e.g. random ordering) reduces performance or not?
4 comments
AuthorsOpenReview ↗

Response to Reviewer eoSd (1/2): W1-W5

We thank the reviewer for the detailed review. We have revised the agent and the experiments (see our general response), and all simulated results below are for the revised version.

W1.

While the authors claim that the formulation can be extended to probabilistic dynamics, in practice they only learn a deterministic simulator. It is not apparent why they did not perform the probabilistic dynamics ablation if their formulation allows them to do so.

Our experiments consider deterministic environments, which are a staple of classic task and motion planning (TAMP). Probabilistic dynamics would not be an ablation: it would be a harder task, and we leave it to future work (footnote in Section 3).

W2.

“Code as world models” idea is not new, and there are many related work in this, such as those mentioned in the paper’s related work section. However, the authors do not demonstrate any comparisons against these baselines apart from ablation studies that are “inspired by” other related work. The results graph is too coarse (only based on five runs) to be able to be considered as reported numbers for any future work to use as comparison points. The benchmark also seems too easy as a task since EMPIRIC saturates all of them.

We know this line of work very well, and none of it learns physical mechanisms as code for robot manipulation. WorldCoder, TheoryCoder, PoE-World and Code World Models are evaluated on grid worlds and games, and POMDP Coder on discrete partially observable tasks; none of them can choose continuous skill parameters, such as a placement pose. We do not claim code world models as our idea. Our contribution is a world-model representation for robots: a domain-agnostic base simulator (a physics engine, a model of the robot, and an interface for adding mechanisms) that the agent extends with learned mechanisms, and a belief over their parameters that supports both learning and planning.

We compare with prior methods wherever they can be adapted to our setting. The baseline that the submission called Standalone sim. is a good-faith adaptation of WorldCoder: with no base simulator, it learns a program that takes an observation and an action (one skill with its parameters) and returns the next observation. We now name it WorldCoder (adapted), and Appendix A.4 states how we adapted it.

The benchmark is no longer saturated. With ten seeds per agent and domain, EMPIRIC solves 47 of 50 runs, and the baselines solve 35 to 41 (Table 1).

W3.

While the authors claim the update mechanism for the code world model to be Bayesian-motivated, in practice only the parameters are fitted Bayesian-style, and the world model proposal and update itself does not take advantage of Bayesian nor gradient descent.

This is correct, and it is a deliberate choice. The Bayesian treatment covers the parameters and the hidden state of a fixed program (Section 3.2), and the coding agent revises the program itself, guided by the fit errors. Gradient descent does not apply here, because the program is discrete code and the simulator is not differentiable.

W4.

EMPIRIC’s ability to “plan” is not demonstrated – with full observation of the geometry in the scene, it is unclear whether EMPIRIC needs to plan and revise its plan at all, since a greedy method may be able to solve it.

Our tasks require planning under uncertainty. The geometry is known, but poses are noisy, quantities such as water temperature and glue bonds are hidden, and mechanism parameters such as a balloon's lift must be learned. On a test task the agent cannot reset, so it must predict the outcome of a plan before it acts: of the 92 failed runs across all agents except the oracle, 90 fail on the test task.

Our baselines are stronger than a greedy method: Direct agent and Direct + scene reason over several steps and replan after each outcome, and they solve 35 and 41 of 50 runs. Rehearsing a plan under the belief helps: the ablation that plans at one parameter setting solves 5 of 10 Domino runs, against 10 for EMPIRIC.

W5.

Both the world model and the evaluation benchmarks are implemented in PyBullet, so there isn’t any evidence that the world model can learn to generalize beyond its prior.

We have evidence beyond PyBullet. In a new study (Appendix A.5), the world runs on MuJoCo while the agent's base simulator stays on PyBullet: EMPIRIC solves 11 of 15 runs, and Direct + scene, the best baseline, solves 9. On the real robot, the agent's model predicted a slide of 10.1 ± 2.3 cm against a measured 9.9 cm. In simulation, the agent learns glue, lift, airflow, water and heat, none of which the base simulator has.

AuthorsOpenReview ↗

Response to Reviewer eoSd (2/2): W6-W7, Q1-Q6

W6.

The agent is given scene geometry as observations, which makes the task trivial, bypassing the need to use the image observations, and undermines the “learnt world model” claim.

Known geometry does not make the tasks trivial. The scene assets give the objects' shapes but no masses, frictions or mechanisms, and at every step the agent observes only noisy object features and images. The baselines and ablations that receive the same assets solve 30 to 41 of 50 runs, and EMPIRIC solves 47 (Table 1).

W7.

The entire paper is clearly heavily written by AI with insufficient human revisions to make it more readable.

We are sorry that the writing was unclear. As the paper states, generative AI assisted the revision, and the authors verified the final text. We have revised the paper for readability, and we are happy to revise any section that you think is still unclear.

Q1.

L118: Doesn’t the environment evaluate the reward R evaluated on the ground truth state, rather than the noise-free observations? If the observation contains the target then the environment setup is limited to tasks with immediate rewards (such as object finding) and not tasks that involves long-term dependencies with delayed rewards.

R reads the true values of the observable object features, which is what we meant by noise-free observations. The setup is not limited to immediate rewards, because R is a binary reward over the whole trajectory: in Domino the target must fall through a cascade, and in Balloons the goal must hold for a period of time. We clarified this in Section 2.

Q2.

L179: If the quantities such as “each glued join’s cure progress” is not kept track of by the engine, how does it know to store these properties in the recurrent state x_res?

The coding agent decides, and the engine plays no part: the agent writes the program, and the program declares its hidden state. The agent is not told which hidden quantities exist, and it finds the need in its experiments. For example, two glued blocks separate in one lift and hold together in another, although they look the same before both lifts, so the agent may add a quantity that records how long the blocks were in contact.

Q3.

Can you add more challenging benchmarks with a more fine-grained success metric that tests the planning capabilities of EMPIRIC?

We added no new benchmark, but the revised setting is harder and no longer saturated: EMPIRIC solves 47 of 50 runs, and the baselines solve 35 to 41. As a finer metric, the appendix now reports the environment steps of the solved runs. Our reply to W4 explains how the tasks test planning.

Q4.

Can you benchmark the performance of EMPIRIC against other existing work (e.g. those listed under “code world models” in the literature review) on benchmarks that are not based on PyBullet?

We compare with this line of work through an adapted baseline, because the systems themselves cannot run on our tasks (see our reply to W2): WorldCoder (adapted) solves 38 of 50 runs, against 47 for EMPIRIC.

For evidence outside PyBullet, we added a study with the world on MuJoCo (see our reply to W5), and the paper also has the real-robot experiment.

Q5.

What is the decision criteria for EMPIRIC to add a new mechanism, as opposed to tuning the existing parameters to fit the new observation?

The criterion is a poor fit. If the best parameter setting still leaves part of the recorded data far from the model's replay, the fit reports it, and the agent then decides whether to run another experiment, refit or add a mechanism (Section 3).

Q6.

While the information gain measure I_j(ω) is mathematically interesting, this quantity is only used to rank experiments, and the LLM agent has the final call, so it is unclear whether this quantity is the optimal choice or simply an intuitively sound choice, and it is also unclear if the LLM agent requires this ranking at all. Could you conduct some ablation studies to verify whether other choices of ranking (e.g. random ordering) reduces performance or not?

On further inspection, we found that the agent rarely used the ranking in our runs (Appendix B.4), so it does not need it, and Section 3.3 now presents I_j as an optional score. The agent still seeks information based on its belief: when a rehearsed plan succeeds under some belief draws and fails under others, the harness names the parameter ranges where it fails, and the agent can run an experiment to narrow them. A rehearsed plan had such a mixed outcome at least once in 49 of the 50 runs.

Reviewer_eoSdOpenReview ↗

Thank you for your response

Thank you for your response and the effort put into the rebuttal. The direction of the improvements you have made to the experiment setting (making the benchmark less saturated, adding baselines, and reporting the numbers in a table) are favorable. However, the benchmark is still almost saturated by the EMPIRIC method, and since there aren't any other natively implemented baselines, it is difficult to judge whether the performance comes from the strength of the method or the simplicity of the benchmark. The AI text problem still is an issue (they tend to have way too many semi-colons and colons, lists items and phrases, and scatters fragmented thoughts rather than construct coherent sentences and link them up properly in a way that flows naturally).

I would like to acknowledge the effort put into the paper and the line of work itself is interesting, so I will increase the rating to 2. Please investigate if there are native baselines you can compare against, or a non-pybullet environment you could use to benchmark your experiments where EMPIRIC won't easily saturate.

AuthorsOpenReview ↗

Response to Reviewer eoSd’s Follow-up Comment

Thank you for reading our response and for raising your score. We answer your remaining points below.

We do not think that the benchmark is simple, as shown by the gap between EMPIRIC and the baselines. All agents use the same coding agent (Claude Opus 5), the same skills, and the same step budget. If the tasks were simple, this coding agent would solve them without our method. Yet it solves 35 of 50 runs on its own and 41 with the scene assets, and it solves only 30 when it has the base simulator but cannot learn mechanisms. EMPIRIC solves 47, so it is close to the ceiling while the baselines are 6 to 17 runs below it. We see this gap as strong evidence for the strength of the method.

You ask for an environment outside PyBullet that EMPIRIC does not saturate. We ran this experiment for the revision (Section 4.2, Appendix A.5). There, the world runs on MuJoCo while the agent's base simulator stays on PyBullet, so the agent can no longer rely on its base simulator to match the physics of the world. EMPIRIC is far from saturated in this setting, with 11 of 15 runs solved, and it is still ahead of Direct + scene, the strongest baseline in the main comparison, which solves 9.

On native baselines, we found no published system that can run on our tasks in its original implementation. The code world model systems (WorldCoder, TheoryCoder, PoE-World, POMDPCoder) choose from a small set of discrete actions and cannot set the continuous parameters of a robot skill, so we adapted WorldCoder to our interface. Methods that learn abstractions for task and motion planning, such as VisualPredicator and ExoPredicator, assume more given knowledge about each domain than our setting provides. Deep reinforcement learning methods can run natively, but we expect them to need far more than our budget of 10,000 to 20,000 environment steps, because the reward is binary and sparse. We will release the domains and the baseline code so that others can run their systems on them.

Finally, we agree that the text still has some limitations, like using too many colons. We are rewriting the main text for the final version.

You wrote that this line of work is interesting. We hope that, with these answers, you will also see the paper as a valuable contribution that can be accepted.

Reviewer_XNbPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces EMPIRIC, a robot agent that learns unfamiliar physical mechanisms by writing them as executable code extensions ("residual world models") into a supplied base physics simulator, inferring their parameters and hidden state from noisy observations, and planning with the extended model over parameterized skills.

Its main contributions are: (1) a residual world-model representation that subclasses a base simulator to add missing mechanisms and recurrent hidden state; (2) approximate Bayesian inference over parameters and state, including uncertainty-aware planning and information-gain-driven experiments using predicates; and (3) execution monitoring with model revision when predictions fail.

The approach is validated on five PyBullet tabletop domains (Domino, Bridge, Balloons, Boil, Fan) against six baselines including oracle dynamics and ablations, where EMPIRIC solves all 25 runs with better sample efficiency, plus a real Franka robot study in which it learns wind dynamics and object masses from two gusts and plans a successful two-domino cascade.

Critical strengths

One concrete strength is the clean, executable residual representation for missing physics, paired with a specific inference procedure. A program P subclasses the base simulator to add only the missing mechanisms and recurrent hidden state while keeping the engine's contact, gravity, and skill dynamics, parameterized by theta. Inference builds a generalized posterior from a replay loss (with temperature, Huber loss, and settled-state weighting), finds the estimate via grid-seeded Levenberg-Marquardt with conditional profiles, and pairs each parameter draw with the recurrent state it implies, using an observation-only posterior for base state. This decouples mechanism revision from parameter fitting, making the model editable and replay-verifiable while supporting uncertainty-aware planning; it underlies EMPIRIC's 25/25 simulated success, matching oracle dynamics, and the real robot fitting wind and mass from two gusts to predict a 10.1 plus or minus 2.3 cm slide against 9.9 cm measured.

Critical weaknesses

The biggest weakness is that the paper's central claim—learning unfamiliar physical mechanisms from few interactions—rests on substantial scaffolding handed to the agent and never learned, so the reported gains confound residual modeling with privileged support. The agent is given scene geometry, a base simulator that already handles contacts, gravity, and robot skills, predefined object features rather than learned perception, a skill library, and a predicate interface, while only a few scalar parameters and a small mechanism hook are truly learned. This matters because the headline baselines lack parts of this scaffolding, so the 25/25 versus 14–16 gap plausibly reflects the supplied base physics and scene more than the residual-learning idea, and ablations show cheap workarounds recover most of the benefit in four of five domains. The single real-robot result reinforces this: one run on one bench, relying on a perception pipeline, calibrated button press, and recovery skills the authors say make the loop executable, with the green domino's mass only lower-bounded. A systematic account of which quantities the robot should decide to infer would be more valuable, though this would likely depend heavily on the capabilities of the underlying LLM or coding agent.

Questions for authors

None.

1 comment
AuthorsOpenReview ↗

Response to Reviewer XNbP

We thank the reviewer for the critical feedback. We have revised the agent and the experiments (see our general response), and all simulated results below are for the revised version.

W1.

The biggest weakness is that the paper's central claim—learning unfamiliar physical mechanisms from few interactions—rests on substantial scaffolding handed to the agent and never learned, so the reported gains confound residual modeling with privileged support.

We agree that the submitted version did not separate what the agent is given from what it learns, and we changed what is provided to our agent and the comparison to address this.

The agent is given 1) scene geometry, 2) a base simulator that already handles contacts, gravity, and robot skills, 3) predefined object features rather than learned perception, 4) a skill library, and 5) a predicate interface, while only a few scalar parameters and a small mechanism hook are truly learned. This matters because the headline baselines lack parts of this scaffolding, so the 25/25 versus 14–16 gap plausibly reflects the supplied base physics and scene more than the residual-learning idea, and ablations show cheap workarounds recover most of the benefit in four of five domains.

Each input that the agent receives is domain-agnostic, shared by all agents in our experiments, or a standard assumption in robot planning. The base simulator (a physics engine, a model of the robot, and an interface for adding mechanisms) and the skill library are the same in every domain. For each domain, EMPIRIC also receives the scene assets (the geometry as URDF and mesh files, with no physical properties like masses or friction) and the object features, and the baselines receive them too (only Direct agent has no scene assets; see Table 1).

Most of each domain's model is learned from the agent's own experiments. The agent chooses which objects' masses and material properties to estimate, writes the code for each missing mechanism and its hidden state, and fits the parameters. In Bridge, for example, the agent's code defines when a block face gets wet, when the glue cures, and which faces bond.

The supplied scene and base physics do not explain the gap, as the new runs show. With the same scene assets, Direct + scene and WorldCoder (adapted) solve 41 and 38 of 50 runs. With the base simulator but no way to add mechanisms, Direct + base simulator and No learning solve 37 and 30, and EMPIRIC solves 47.

Each part of the learning workflow contributes. The two ablations in question remove only the harness fit or the explicit uncertainty, and they solve 41 and 39 of 50 runs. In Domino, where a single push that cannot be undone decides the outcome, they solve 5 of 10 runs each, against 10 of 10 for EMPIRIC.

The single real-robot result reinforces this: one run on one bench, relying on a perception pipeline, calibrated button press, and recovery skills the authors say make the loop executable, with the green domino's mass only lower-bounded.

The real-robot experiment demonstrates that the approach applies in the real world, and we draw no statistical claim from this one run. The agent solved the task with real perception and real contact, and its model predicted a slide of 10.1 ± 2.3 cm against a measured 9.9 cm. A perception pipeline and calibrated skills are standard parts of a robot system.

We also added a study in which the world's physics differ from the agent's base simulator, as they do on a real robot. The world runs on MuJoCo and the base simulator stays on PyBullet: EMPIRIC solves 11 of 15 runs, and Direct + scene, the best baseline of the main experiments, solves 9 (Appendix A.5).

A systematic account of which quantities the robot should decide to infer would be more valuable, though this would likely depend heavily on the capabilities of the underlying LLM or coding agent.

The agent's choice of quantities is tested against data in every run: a poor choice appears as a poor fit or a failed prediction, and the agent can then revise the program. We agree that this choice likely depends on the underlying model, and all our agents use Claude Opus 5.

Comments on the paper

2 comments

The author-reviewer discussion period closes on November 18 (AoE). Please engage during this period so that we have our issues discussed thoroughly.

AC

AuthorsOpenReview ↗

General Response: Revised Agent and New Experiments

We thank the reviewers for their careful and critical feedback. We are encouraged that all three reviewers learned something interesting or useful from the paper, and by their positive comments on the main idea: a clean and executable representation of missing physics (XNbP) that uses both a physics simulator and a learned model (eoSd), and a practically meaningful design (M7Ty). Our contribution is this world-model representation for robots: the agent extends a domain-agnostic base simulator with mechanisms that it learns from its own experiments, and it plans under a belief over their parameters (eoSd W2, M7Ty W1). We made four main changes, and all simulated results below are for the revised version.

  1. EMPIRIC now starts from one domain-agnostic base simulator (a physics engine, a model of the robot, and an interface for adding mechanisms), prompt and tool set, and learns each domain's mechanisms and parameters from its own experiments (XNbP W1, M7Ty W1, M7Ty Q3). In this harder setting, it still solves the most runs: 47 of 50, and the four baselines solve 35 to 41 (Table 1).
  2. We ran ten seeds per agent and domain, 450 runs in total (eoSd W2). EMPIRIC solves at least 8 of 10 runs in every domain.
  3. We added a baseline and an ablation (Direct + base simulator and No learning) that separate the supplied simulator from what the agent learns, and every baseline now has the task evaluator (XNbP W1, M7Ty W2, M7Ty Q1). The two solve 37 and 30 of 50 runs, so the supplied simulator alone does not explain EMPIRIC's 47.
  4. In a preliminary study, the world runs on a different physics engine (MuJoCo) than the agent's base simulator (eoSd W5, eoSd Q4). EMPIRIC solves 11 of 15 runs, and the best baseline from our main experiments (Direct + scene) solves 9.

The new runs do not support our earlier claim that EMPIRIC uses fewer environment steps, and we removed it (Appendix A.3).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.