acceptodds
Under review as a conference paper at ICLR 2027

Schema: Discovering Unknown Environments via Agentic Program Induction

14 views

Abstract

Learning to complete tasks in unfamiliar environments with unknown rules remains a key challenge for LLM agents. Current LLM agents often record their discoveries in prose, which may not provide a compact, explicit account of how the environment works. Inspired by how scientists organize observations into testable, predictive theories, we introduce Schema, an agent harness that organizes learning and action through interactive program induction. The LLM agent decides what to investigate and how to act, expressing its evolving understanding of the environment as executable programs. The harness consists of a persistent program workspace and a small set of interfaces for checking these programs against the interaction history, planning within them, and executing plans under step-by-step verification. Schema raises ARC-AGI-3 RHAE from 58.7% to 99.2% with the same base model, solves 100% of the public DiG-bench games, and reaches the median performance of the top-50 human players on MazeBench. Extensive analysis shows the effectiveness of Schema in unknown mechanism discovery, and ablations confirm the contribution of each component.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Agentic Harness, Program Induction

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_7TCMOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes SCHEMA, an agentic framework that aims to explore, understand, and solve tasks in environments with unknown transitions and goals. It achieves this through a four-stage pipeline. First, it uses a Python program to ground the state into structural representations and hypothesize the environment transitions. Next, it checks whether the hypothesized transition functions match the recorded historical transitions and revises them if not. Then, it asks the agent to come up with a target to either explore the environment or solve the task, and use planning in this Python-form dynamic model to achieve it. Finally, it executes the plan in the real environment, where the agent will stop to replan whenever the environment transition conflicts with the Python one. It achieves superior performance on ARC-AGI-3, DiG-bench, and MazeBench, where the ablations show the necessity of each component in the pipeline.

Critical strengths
  • The motivation for using Python programs as abstractions to hypothesize the unknown environments is well-motivated and discusses an interesting direction in Agentic problem solving.
  • The paper is generally well-written and easy to follow.
  • The experimental results demonstrate superior performance over baselines, especially on ARC-AGI-3, showcasing the usefulness of the proposed SCHEMA.
Critical weaknesses
  • Some details of the proposed method are missing:
    • The prompts or prompt design are missing. Since the paper aims to solve tasks in an unknown environment, it's important to ensure the prompts do not contain any human expert knowledge about the environment.
    • The criteria for whether the transition function is ready to be planned are not provided. Since the authors report that the certification rate may not always be 100%, how does it ensure the certification process checks all the most critical transitions in the environment? (E.g., if the criteria is over 99%, and some transitions are rare in the history, ignoring them may still pass the certification stage)
  • Details of the baselines are missing, e.g., what is the basic harness in ARC-AGI-3? Do they have a memory system? What's the difference between SCHEMA and these baselines? Including these details can make the paper more promising and self-contained.
  • The paper misses a limitations section, where the current evaluation protocol focuses on text-based, discrete-action, deterministic environments. Including it would better help readers understand the scope of the current method.
Questions for authors
  • What are the details of the prose model variant in Table 1? Does it still do some kind of planning, but in text form?

The following questions are out of the reviewer's curiosity and will not affect the score.

  • What if the environment provides only a dense reward? E.g., the reward is given based on the distance to a possibly unknown target; how does the LLM encode it into a transition function without knowing the true nature of the reward?
  • The current evaluation protocol follows an online setting; that is, it 'trains' the program and tests it in the same environment. It will be interesting to evaluate the program in other environments with the same transitions in a zero-shot manner to see if the program just overfit on the training environment.
  • What if the environment is not deterministic? How do we evaluate the correctness of the transition function in this case?
7 comments
AuthorsOpenReview ↗

Response to Reviewer 7TCM [1/5]

We sincerely thank reviewer 7TCM for their detailed feedback. We would like to address their questions as follows:

The prompts or prompt design are missing. Since the paper aims to solve tasks in an unknown environment, it's important to ensure the prompts do not contain any human expert knowledge about the environment.

We thank the reviewer for the suggestion. In Appendix B of the revision, we added the system prompt used by Schema, as well as the tool specifications.

These prompts contain no game-specific rules, completion conditions, meanings of screen elements, or hints about mechanisms. They only contain general working principles, tool usage, and the necessary information about the environment interface. All environment-specific mechanisms are discovered by the agent itself through interaction with the environment.

The criteria for whether the transition function is ready to be planned are not provided. Since the authors report that the certification rate may not always be 100%, how does it ensure the certification process checks all the most critical transitions in the environment? (E.g., if the criteria is over 99%, and some transitions are rare in the history, ignoring them may still pass the certification stage)

We thank the reviewer for raising this question. Schema does not use a rule like "planning is allowed once history accuracy exceeds a certain threshold". The system prompt instructs the agent to have the program reproduce all recorded transitions before planning, but the harness does not make 100% agreement a mandatory gate for acting. Whether to keep revising the program or to first collect new evidence through interaction is decided by the agent, depending on the problem at hand. In practice, the agent usually plans after the history is fully reproduced, and it also keeps exploring when a few transitions cannot be explained for the moment.

Rare but critical transitions are not hidden by the overall accuracy. backtest reports each mismatched historical transition individually, with its index, so that the agent can inspect the corresponding prediction and the real observation. Transitions that cannot be explained yet stay in the history, and as long as the mismatch is unresolved, it is reported again in later full backtests.

In addition, the harness still verifies the program's predictions step by step when executing actions. As soon as the actual observation disagrees with the prediction, it immediately stops the remaining actions and returns that transition to the agent, which can then continue the investigation using both the new and the old evidence.

Explaining an existing mismatch sometimes requires new evidence from interaction. If all mismatches had to be eliminated before further interaction were allowed, the agent could lose the chance to get this evidence. Therefore, Schema allows the agent to leave unexplained transitions open for a while, keep running experiments to collect new evidence, and then revise the model accordingly and check it against the full history.

Here, we would like to share two interesting observations related to this.

  • A new mechanism can explain an old error in hindsight: in SU15 level 3, the agent could not yet explain a transition in which a block was shrunk and knocked back by a hunter. Clearing level 5 requires this mechanism, and through repeated experiments the agent found that it is triggered when the block blocks the hunter's movement. The updated program also correctly reproduces the transition left over from level 3 (Appendix I of the revision, Figure 24).

  • An unresolved mismatch can serve as a clue: in KA59 level 5, the agent could not find a route over the wall, so it went back to the mismatch left over from level 4 and re-examined its assumption that "the timer only pushes pieces downward". It ran an experiment with a piece placed above the timer and found that the piece can be launched upward. It then corrected the direction rule and used this mechanism to plan how to clear the level (Appendix I of the revision, Figure 25).

We have made the role of certification clearer in Sec 3.2 of the revision.

AuthorsOpenReview ↗

Response to Reviewer 7TCM [2/5]

Details of the baselines are missing, e.g., what is the basic harness in ARC-AGI-3? Do they have a memory system? What's the difference between SCHEMA and these baselines? Including these details can make the paper more promising and self-contained.

Thank you for the suggestion. At the end of Appendix B in the revision, we added full descriptions of both types of baselines (Table 3 summarizes how they differ from Schema), and added a brief description in Sec 4.1 of the main text. All harnesses use the same base model and the same action budget.

Basic harness. We use the official ARC-AGI-3 basic harness (the Standard harness that ARC Prize uses to evaluate frontier models). At each turn, the model sees the game state, the number of completed levels, the current frame (a numeric grid in text form, including animation frames), and the available actions. The system prompt asks it to write down in its reply any information it wants to keep and to give an action. The last action mentioned in the reply is executed in the next turn, and only one action is executed per turn. The harness provides no tools, so the model cannot run code or read or write files. Its memory is a rolling conversation window: each turn's observation and reply are appended to the conversation, and the oldest turns are dropped once the limit is exceeded (about 175K tokens, estimated from character count). If the model wants to keep a finding for the long term, it can only do so by writing it down again and again in later replies. There is no other memory system.

Coding harness (Claude Code / Codex). The basic introduction to the game in the system prompt, as well as the observation and action protocols, are essentially the same as in Schema. The agent keeps all the native coding tools of the CLI (shell, reading/writing/editing files, search). The model can freely keep persistent notes in its working directory, save its history, and write and run code. Its memory is therefore the conversation context plus the files it writes itself.

Differences from Schema.

Basic harnessCoding harnessSchema
World modelContext onlyNotes or codeExecutable program
CertificationNoneCustom scriptsFull-history backtest
PlanningNoneCustom searchBFS/A*
ExecutionOne actionAction sequencesVerified sequences

Verified execution stops the sequence at the first prediction mismatch.

A coding harness can also write its own simulator, history replay script, and search program. Schema lets the same executable program directly support history backtesting, search-based planning, and step-by-step verified execution, while the harness maintains the interaction history automatically. The agent decides on its own when to revise the model, check it, plan, or keep exploring. The ablations in Table 1 keep the general-purpose tools and remove each of these mechanisms separately, which directly tests their contribution to performance.

The paper misses a limitations section, where the current evaluation protocol focuses on text-based, discrete-action, deterministic environments. Including it would better help readers understand the scope of the current method.

We thank the reviewer for the suggestion. We added a "Limitations and Future Work" section at the end of the main text in the revision. It discusses the challenges of extending to more complex environments, including learning mechanisms from raw real-world perception, handling stochastic events, and planning in continuous action spaces.

AuthorsOpenReview ↗

Response to Reviewer 7TCM [3/5]

What are the details of the prose model variant in Table 1? Does it still do some kind of planning, but in text form?

Thank you for the question. We observe that the prose-model agent still checks its history, simulates its action plans, and compares execution results with what it expected, but it does all of these through natural-language reasoning.

This variant uses the same base model and interaction budget as Schema. It keeps the general-purpose tools and history access, and maintains its world model as written rules in notes.md. Since there is no executable prediction model, we removed the tools that depend on one: backtest, model_predict, and run_bfs. Action sequences can still be committed and executed, but execution cannot stop automatically when a prediction is wrong.

  • Certification: the agent uses read_history to pull up past transitions and checks them by working through the rules in notes.md itself. It does 185 such checks in total, covering about 1,600 transitions. On manual review, we found that some of these checks missed errors in the rules. Schema, in contrast, actually runs the program to produce predictions; the tool compares the predictions with the real observations one by one and returns the mismatched transitions along with the specific differences.

  • Planning: the agent simulates candidate action sequences in natural-language reasoning, following the rules in notes.md. Of the 2,832 action sequences it commits, about 50% contain predicted outcomes three or more steps ahead, and 58% state the expected outcome after the whole sequence. In simple cases, it can get correct routes of a dozen or so steps this way.

  • Verified execution: the agent looks at the returned frames after the whole action sequence has run and compares them with what it expected. By committing shorter sequences, it gets environment feedback and checks its expectations more often, which approximates online verification during execution. In practice, it commits 3.7 steps per sequence on average, compared with 15.7 for Schema.

Reviewer_NQuZOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces an agent harness called Schema which helps LLMs learn unfamiliar environments by constructing programs that define the rules of the environment. The agent checks the program against record interactions (history) and uses them to plan and do actions. If the outcomes of the actions differ from the predictions, the LLM revises the program. The paper does experiments on ARC-AGI-3, DiGBench and MazeBench and demonstrates substantial improvements over baseline harnesses with frontier models. It also includes ablations and analyses on how each component help the agents find new mechanisms in a environment and make progress over long horizons.

Critical strengths

The key contribution is having agents generate Programs that define the environment, having historical checking to verify the programs are grounded in the environment, planning actions for the environment and having runtime verification to provide feedback back to the agent.

Strengths

  1. The paper demonstrates substantial improvements over other basic and coding agent harnesses across multiple environments and models. On ARC-AGI-3, Schema improves RHAE by 30-40+ percentage points across frontier models like Claude Opus 4.8, Claude Fable 5, GPT 5.6 Sol xhigh and GPT 5.6 Sol max (Figure 3).
  2. The ablations and analyses about each component help explain the improvements. They mention how the generating programs helps reduce the number of actions it takes to solve a problem as is the case with CN04 Level 5 finishing in 279 actions (Section 4.1, pg. 6). In AR25, a code revision without a certificate reduces the historical prediction agreement from 97.7% to 65.2% (Section 4.1, pg. 6). In LS20, planning step reduces actions from 233 to 65 even with passed historical checks (Section 4.1, pg. 6). They further mention runtime verification improves adaptation to new scenarios which is also justified with experiments (Section 4.1, pg. 6).
Critical weaknesses
  1. The experiments compare Schema with basic and coding harnesses, but there are no comparisons with program-based methods mentioned in Section 5, such as WorldCoder.
Questions for authors
  1. It would be interesting to see how program-based methods such as WorldCoder do the tasks using the same models and resource? Could you provide more insights on why Schema is potentially better?
  2. With the prose model, could you provide some insights on if that variant utilize certification, planning and/or verified execution? Please explain this ablation a bit more.

Overall, I believe this is a strong accept as Schema demonstrates substantial improvements and takes inspiration from previous program-based methods and gives agent most of the control. A comparison with other program-based methods would strengthen the argument for the paper.

5 comments
AuthorsOpenReview ↗

Response to Reviewer NQuZ [1/3]

We sincerely thank reviewer NQuZ for their detailed feedback. We would like to address their questions as follows:

The experiments compare Schema with basic and coding harnesses, but there are no comparisons with program-based methods mentioned in Section 5, such as WorldCoder. It would be interesting to see how program-based methods such as WorldCoder do the tasks using the same models and resource? Could you provide more insights on why Schema is potentially better?

Thank you for the suggestion. We added a comparison between WorldCoder (Tang et al., 2024) and Schema on the 10 hardest ARC-AGI-3 games used in the Table 1 ablations. Both methods use Claude Opus 4.8 (max effort), and each game gets 8 hours of effective wall-clock time. The full implementation details, per-game results, and analysis are now in Appendix L of the revision (Table 16), and are referenced in Sec 4.1 and Related Work.

Implementation. We adapted WorldCoder to ARC-AGI-3 following Algorithm 1 of the original paper:

  • When the program is inconsistent with the recorded data, or is "not optimistic" in the current state (the planner cannot find a path that clears the level), REx refinement search is called;

  • Thompson sampling selects the program to revise, and the LLM revises it based on counterexamples;

  • If the program is already consistent but not optimistic, the LLM is asked to propose an optimistic hypothesis that makes the current level solvable while staying consistent;

  • 5% random exploration;

  • Otherwise, it plans in the program and executes the plan, comparing every step with the program's prediction; when no plan is found, it explores randomly.

Differences from the original: (1) the original takes 10 random steps before learning starts (MinDataSize=10); we instead try each legal action once first; (2) the original planner is value iteration / MCTS; we replaced it with Schema's BFS/A*.

WorldCoder and Schema share the same program interface, history consistency check (backtest), and BFS/A* planner. The main difference between the two is how learning and interaction are controlled. WorldCoder calls the LLM as an operator that generates and revises code inside a fixed learning procedure. Schema gives the LLM general principles and interfaces, and lets the LLM agent decide for itself when to propose hypotheses, which experiments to run, and when to check and plan.

Results. Each game again gets 8 hours of effective wall-clock time (including LLM inference, tool calls, and planning):

Levels cleared / RHAE (%).

GameWorldCoderSchema
WA300 / 0.02 / 3.4
LF521 / 1.83 / 10.9
AR251 / 2.88 / 89.9
CN040 / 0.04 / 46.4
DC220 / 0.00 / 0.0
G50T0 / 0.04 / 35.7
M0R00 / 0.04 / 46.8
RE861 / 2.65 / 41.7
SK480 / 0.01 / 1.6
LS201 / 3.05 / 47.6
Total / mean4 / 1.036 / 32.4

(For Schema, we take the first 8 hours of effective runtime from its Table 1 run; with the full 2,000 steps, Schema's final score is 72.9.)

We further looked into WorldCoder's reasoning traces. Two representative failure modes are:

  • In LF52, to fit the entire past history, the program creates a global list variable called "MEMORIZED" and then hardcodes all past counterexamples into it.

  • In DC22, the program has to satisfy the optimism requirement, i.e., some way to clear the level must be reachable. So after none of the 9 positions in the box triggered level completion, it attached extra conditions to these positions, such as "whether the pattern on the right has changed", and made these combinations new completion conditions only because they had not been tried yet.

A fixed synthesis-and-acceptance procedure easily leads the program to overfit the limited interaction record, meeting the requirements by memorizing counterexamples and piecing together completion conditions. Schema instead gives model revision, checking, planning, and environment interaction to the agent as tools, and lets it freely decide what to verify, which experiments to run, and when to update the model, so that it can discover the general mechanisms and the true completion conditions.

In addition, every WorldCoder revision requires regenerating the complete program. Repeated synthesis takes a lot of time and leaves fewer chances to collect new evidence from interaction.

AuthorsOpenReview ↗

Response to Reviewer NQuZ [2/3]

With the prose model, could you provide some insights on if that variant utilize certification, planning and/or verified execution? Please explain this ablation a bit more.

Thank you for the question. We observe that the prose-model agent still checks its history, simulates its action plans, and compares execution results with what it expected, but it does all of these through natural-language reasoning.

This variant uses the same base model and interaction budget as Schema. It keeps the general-purpose tools and history access, and maintains its world model as written rules in notes.md. Since there is no executable prediction model, we removed the tools that depend on one: backtest, model_predict, and run_bfs. Action sequences can still be committed and executed, but execution cannot stop automatically when a prediction is wrong.

  • Certification: the agent uses read_history to pull up past transitions and checks them by working through the rules in notes.md itself. It does 185 such checks in total, covering about 1,600 transitions. On manual review, we found that some of these checks missed errors in the rules. Schema, in contrast, actually runs the program to produce predictions; the tool compares the predictions with the real observations one by one and returns the mismatched transitions along with the specific differences.

  • Planning: the agent simulates candidate action sequences in natural-language reasoning, following the rules in notes.md. Of the 2,832 action sequences it commits, about 50% contain predicted outcomes three or more steps ahead, and 58% state the expected outcome after the whole sequence. In simple cases, it can get correct routes of a dozen or so steps this way.

  • Verified execution: the agent looks at the returned frames after the whole action sequence has run and compares them with what it expected. By committing shorter sequences, it gets environment feedback and checks its expectations more often, which approximates online verification during execution. In practice, it commits 3.7 steps per sequence on average, compared with 15.7 for Schema.

AuthorsOpenReview ↗

Response to Reviewer NQuZ [3/3]

To test whether natural-language reasoning can plan effectively once the rules are given, we ran an additional controlled experiment on 15 ARC-AGI-3 levels. All conditions use the program that Schema finally learned in the corresponding game as the source of rules, and we compare three ways of planning:

  • Search: Schema's BFS searches for a path in the program;

  • Reasoning (code): Opus 4.8 (max effort) reads the program but cannot run it, and writes an action sequence by reasoning alone;

  • Reasoning (text): Opus 4.8 first rewrites the program as plain-text rules, and then reasons from these text rules only.

All plans (including those found by search) are executed in the real game, and a plan succeeds only if it clears the level in the real environment.

LevelStates expanded by searchSearchReasoning (code)Reasoning (text)
AR25 L1475✓ 15 steps, 0.8 s✓ 15 steps, 2.2 min✓ 15 steps, 1.9 min
AR25 L267✓ 11 steps, 0.2 s✓ 11 steps, 4.8 min✓ 11 steps, 4.2 min
AR25 L5181k✓ 28 steps, 445 s✓ 28 steps, 22.6 min✓ 28 steps, 9.9 min
RE86 L3236✓ 47 steps, 0.1 s✓ 47 steps, 4.3 min✓ 47 steps, 5.3 min
RE86 L5386✓ 63 steps, 0.2 s✓ 63 steps, 13.4 min✓ 63 steps, 26.5 min
M0R0 L113k✓ 15 steps, 8 s✓ 15 steps, 11.3 min✓ 15 steps, 24.3 min
M0R0 L437k✓ 11 steps, 33 s✗✓ 25 steps, 22.8 min
DC22 L124k✓ 20 steps, 22 s✓ 20 steps, 7.7 min✓ 20 steps, 24.9 min
SK48 L1691✓ 14 steps, 0.9 s✗✗
LS20 L1485✓ 13 steps, 0.3 s✓ 13 steps, 20.3 min✗
LS20 L25.7k✓ 45 steps, 4 s✗✗
LS20 L318k✓ 39 steps, 12 s✗✗
LS20 L427k✓ 43 steps, 17 s✗✗
LS20 L5304k✓ 44 steps, 265 s✗✗
LS20 L720k✓ 53 steps, 138 s✗✗

(✗ means the level was not cleared.)

  • Search clears all 15 levels; reasoning (code) clears 8, and reasoning (text) clears 8.

  • On simple levels, reasoning can also find correct plans, usually with about as many steps as search. But producing one plan takes from a few minutes to tens of minutes and uses more tokens, while search mostly finishes within seconds. In this diagnostic experiment, explicit search is usually faster and more reliable.

  • On levels that need more search, reasoning often fails to find a correct plan. For example, some LS20 levels require expanding thousands to 300,000 states: search clears all 6 tested levels, reasoning (code) clears only level 1, and reasoning (text) clears none.

So the prose variant also performs certification and planning through natural-language reasoning, and approximates online verification during execution, but it is less efficient and less accurate than an implementation based on executable programs. This explains its performance gap to Schema.

The details of the prose-model variant and the planning experiment above are now in Appendix H of the revision (Table 13).

Reviewer_9AH1OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents Schema, an LLM agent harness for learning to act in unfamiliar environments. The agent writes an executable program to represent its current understanding of the environment, checks the program against recorded interactions, plans by simulating the program, and verifies predicted outcomes during execution. The base LLM is frozen. The main contribution is an agent-directed workflow linking programme revision, history-based checking, planning, and interaction.

The paper reports strong results on the 25 public ARC-AGI-3 games, 21 public DiG-bench games, and a long MazeBench run. The large gains over basic and coding harnesses are interesting. The ablations also suggest that the four components of Schema are practically useful.

My overall view is positive about the empirical system, but more reserved about the scientific claims. In particular, successful interaction and a growing executable codebase are not sufficient evidence that the agent has learned reusable, generalisable world models. The paper should analyse the content and transfer of its learned programs more directly. I think this can be addressed through targeted evaluation and clearer positioning, without needing to redesign the whole system.

Critical strengths
  1. The problem is important. Learning unknown environment mechanisms through interaction is a meaningful challenge for LLM agents. A persistent executable representation is a promising way to support prediction and planning.

  2. The empirical improvements are substantial. The same-base-model comparisons, especially against coding harnesses, make the results worth taking seriously. The DiG-bench repeated runs and the component ablations are useful.

  3. The paper gives concrete behavioural examples, rather than only final scores. The history-checking failure in AR25, the planning example in LS20, and the ice-sliding rule reused in another MazeBench room help explain how the system operates.

  4. I appreciate that the authors discuss WorldCoder, CWM, TheoryCoder, and related executable world-model work. This provides a basis for a more precise statement of the paper's contribution.

Critical weaknesses

1. The evidence for reusable and generalisable world models is still limited

This is my main concern. The paper claims that Schema discovers environment mechanisms and accumulates them as reusable executable knowledge. However, most reported metrics measure task completion, action efficiency, code size, or agreement with observed history. These are important system metrics, but they do not establish whether the resulting programs represent shared abstractions rather than collections of task-specific fixes.

For example, Figure 9(c) reports call sites per function. This measures static references in the code, not the number of distinct rooms, levels, tasks, or mechanisms in which a function is actually used. An increase in call sites could occur even if the program contains many ad hoc functions for individual rooms. Similarly, the number of recorded transitions explained per line is not a direct measure of abstraction quality.

Appendix I provides a valuable example: an ice-sliding mechanism learned in one room is reused much later in another, with the main functions and classes unchanged. This supports a specific case of transfer. It does not yet establish how common such transfer is across the learned codebase.

I suggest a systematic programme-level analysis. The authors could label functions or modules by their role (state grounding, transition dynamics, task-specific logic, and planning), then report how many distinct rooms, levels, and mechanism instances use each one without modification. Runtime invocation counts and a function-by-task reuse matrix would be more informative than static call sites alone. A distribution is important here, not only a few selected examples. It would also help to report how much code is genuinely shared and how much is added specifically for each new room or game.

An especially useful experiment would be to freeze the learnt grounding and transition code after encountering a mechanism, and test it on new layouts or object configurations involving that mechanism. The authors could report prediction accuracy and planning success before any further code revision. This would distinguish transfer of a learned model from successful reprogramming on the new task.

2. The paper does not directly evaluate the quality of the induced grounding and transition abstractions

Section 3.1 describes two different parts of the program: state grounding and transition rules. This distinction is important, because a model can have correct action rules but poor object grounding, or can solve a level through hardcoded state-specific conditions.

However, the evaluation does not separately measure the correctness, stability, or reuse of the grounding functions. Nor does it systematically test the induced transition rules on independently sampled, unobserved states. History certification, defined in Equation (2), only establishes agreement with previously recorded interactions under the paper's observation-equivalence criterion. It does not by itself establish predictive validity beyond that history.

I suggest reporting held-out one-step transition prediction accuracy, evaluated on states not used to write or revise the program. The experiment should separate previously observed mechanism types from unseen configurations of those mechanisms. Where possible, errors should be broken down into grounding errors and dynamics errors. It would also be useful to report how often previously learnt functions must be changed when the agent encounters a new configuration.

The programme structure itself deserves analysis. Do the learnt models contain reusable object types, predicates, and relations? Are these combined to handle unfamiliar configurations, or are there many branches tied to room identifiers and particular visual patterns? A small, systematic code audit across several games would provide strong evidence either way.

3. Evidence for compositional generalisation and hierarchical goal abstraction is missing

The paper allows the agent to specify a goal predicate and search within its program (Section 3.3). But choosing a goal for a planning call is not the same as learning and reusing higher-level goal abstractions or hierarchical task structures.

I did not find a systematic analysis of learnt subgoals, reusable options, task decompositions, or their transfer across levels. I also did not find a controlled evaluation of novel combinations of previously learned mechanisms.

A useful test would teach the agent two mechanisms in separate contexts and evaluate a new task requiring their combination. The relevant grounding and transition modules should remain frozen during the transfer test. For hierarchical claims, the authors could show explicit goal or subgoal structures extracted from the agent's workspace, and measure how often these are reused across genuinely different tasks. If hierarchical goal abstraction is not a contribution of Schema, the text should say so clearly rather than imply it from general planning performance.

4. The novelty claim needs more careful positioning against prior code world models

The method is effective, but its main operations have substantial precedents. LLM-written world models, counterexample-based programme revision, model-based planning, and prediction monitoring are all established ideas. The paper's actual difference appears to be that a general-purpose LLM controls when and how to perform these operations through a common harness, including revisions to the state representation.

This is a legitimate architectural contribution. However, terms such as a new paradigm and the claim of being the first harness to achieve human-level mechanism discovery are rather strong. I would prefer the paper to specify which learning or transfer capability Schema adds beyond WorldCoder, CWM, and TheoryCoder, and which gains arise from the integration of familiar components.

The existing component ablations are helpful, but comparisons with a basic harness or a general coding harness do not isolate the advantage over prior executable-world-model agents. A matched implementation of a prior model-learning-and-planning loop, or a more tightly controlled fixed-schedule variant using the same Schema tools, would help test the contribution of agent-directed control. The paper should also distinguish the scientific claim about mechanism learning from the engineering claim about better game performance.

5. Strong headline scores need careful qualification and comparable evaluation

The ARC-AGI-3 results are impressive, but they are obtained on the public games. The paper's own Appendix C.1, Table 2, also lists GPT-6 Astra with a provider-adapter harness achieving 99.9% RHAE on the semi-private evaluation. This is a different set of games, so the figures are not directly comparable. Nevertheless, it is relevant context for broad statements about being the first to attain human-level results in unknown environments and for attributing such performance uniquely to Schema.

I suggest that the authors clearly distinguish benchmark splits, base models, harnesses, and reported scoring protocols in all headline claims. Where possible, a same-model, same-split comparison with a strong provider-adapter or model-based harness would be valuable. The paper should also report inference cost and tool-execution cost alongside environment actions. The DiG-bench cost-per-cleared-level analysis is a good start, but similar reporting would help for the other settings.

Finally, the paper provides repeated runs on DiG-bench, while the long MazeBench result is essentially a trajectory-level case study. I would avoid making broad statistical claims about reliable long-horizon abstraction from that single trajectory. The individual case is informative, but its representativeness is unknown.

Questions for authors
  1. What exactly constitutes a reusable mechanism in the learned program? Can the authors identify its grounding module, transition module, and scope of application? How many distinct rooms or levels invoke each module without changes?

  2. How much of the codebase is shared versus task-specific? Please consider reporting runtime function-use statistics, a function-by-task reuse matrix, code additions per new environment, and the fraction of branches conditioned on specific room or level identifiers. Static call-site counts alone are insufficient.

  3. How well does the frozen model generalise? Once a mechanism has been learned, can the authors test its one-step predictions on held-out configurations before permitting additional code edits? A frozen-model transfer experiment would be more direct evidence for reusable world modelling than final task success alone.

  4. Does Schema learn grounding abstractions as well as dynamics? Can the evaluation distinguish grounding failures from transition-rule failures? How often are grounding functions transferred unchanged to new objects, layouts, or rooms?

  5. Is there evidence of compositional or hierarchical reuse? Can two mechanisms learned separately be applied together in a new task without a newly hardcoded combined rule? Are subgoals or reusable task decompositions represented in the workspace, and how often are they reused?

  6. Can the role of agent-directed control be isolated? How would Schema compare with a fixed schedule of programme revision, history checking, and planning using the same tools and model? A direct comparison with a prior executable-world-model method would make the novelty claim more convincing.

  7. How should interaction cost be interpreted? I would not treat a large number of exploratory actions or failed predictions as evidence of poor abstraction by itself. A careful agent may conduct more experiments to test uncertainty. It would be useful to distinguish deliberate informative probes from unexpected prediction failures, and to analyse whether a learned mechanism reduces model revision, rather than simply action count, when encountered again. Such statistics would complement, but should not replace, frozen-model transfer tests.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 9AH1 [1/4]

We sincerely thank reviewer 9AH1 for their detailed feedback. We would like to address their questions as follows:

1. Reusability and predictive generalization (W1, W2, Q1–Q4, Q7)

The paper claims that Schema discovers environment mechanisms and accumulates them as reusable executable knowledge. However, most reported metrics measure task completion, action efficiency, code size, or agreement with observed history. These are important system metrics, but they do not establish whether the resulting programs represent shared abstractions rather than collections of task-specific fixes.

For example, Figure 9(c) reports call sites per function. This measures static references in the code, not the number of distinct rooms, levels, tasks, or mechanisms in which a function is actually used.

An especially useful experiment would be to freeze the learnt grounding and transition code after encountering a mechanism, and test it on new layouts or object configurations involving that mechanism. The authors could report prediction accuracy and planning success before any further code revision.

I suggest reporting held-out one-step transition prediction accuracy, evaluated on states not used to write or revise the program.

We thank the reviewer for the suggestions. We added two systematic analyses: (1) we freeze the complete program and test how well it predicts unseen levels, with no code edits allowed; (2) we audit the structure of the MazeBench program across rooms, to check whether different rooms are handled by shared mechanism code. The results show that the programs learned by Schema not only explain the observed history but also predict unseen levels, and that they handle different rooms through shared mechanism code. This gives direct evidence that environment knowledge is accumulated and reused.

Frozen-program prediction on new levels (ARC-AGI-3). We freeze the complete program saved when the agent clears level L−1 and use it to predict the transitions of level L in the same game. The code is not modified at any point during evaluation. The program first receives the initial frame of level L and the level index, which is the same initialization as when the agent normally enters a new level. It then makes offline one-step predictions along the trajectory actually recorded on level L. At each step, it takes the true current frame and the executed action as input, while its internal hidden state is advanced by its own predictions.

Level L has a different layout from the previous levels and usually introduces new mechanisms. None of its transitions were used to write or revise the program, so this is a strict held-out prediction evaluation. Changed-cell accuracy is the fraction of cells that actually change whose predicted color exactly matches the true color; unchanged-cell accuracy is computed in the same way on the cells that do not change. The statistics cover all main-experiment runs of the four models, more than 500 levels in total.

ModelChanged acc.L1 controlUnchanged acc.Code retained
Claude Opus 4.862.9%31.6%98.5%83.0%
Claude Fable 564.9%34.6%98.8%79.7%
GPT-5.6 Sol (xhigh)63.4%27.3%97.1%77.4%
GPT-5.6 Sol (max)61.2%30.7%98.5%79.2%
Average63.1%31.1%98.2%79.8%

L1 control reports changed-cell accuracy using the level-1 program.

  • Even though level L may introduce mechanisms the program has never seen, the frozen program still correctly predicts 63.1% of the changed cells, while keeping 98.2% unchanged-cell accuracy.

  • On the same unseen levels, the program that has only seen level 1 correctly predicts just 31.1% of the changed cells. Later programs predict about twice as well as early ones. This directly shows that, as interaction goes on, the program accumulates predictive knowledge that transfers to new layouts, rather than just fitting the observed history more closely.

  • Adapting to a new level only takes incremental changes: about 80% of the code in the program that clears level L already existed when the agent entered that level, and was kept.

This result is now in Sec 4.1 of the revised paper (Table 2). The evaluation protocol, and a case study of how the AR25 program evolves level by level, are in Appendix M of the revision.

AuthorsOpenReview ↗

Response to Reviewer 9AH1 [2/4]

Program structure audit (MazeBench). On the reviewer's point that call sites only reflect static references, we audited the program from the long MazeBench run in the paper (GPT-6 Astra, 139 rooms visited).

The program has two parts: a set of mechanism functions shared by all rooms (69 functions and about 3,000 lines in the final program), and static geometry data stored per room (layout, heights, and materials).

In the final program and in all 33 program snapshots saved each time a gem was collected, no mechanism function contains a room identifier, i.e., the number of branches on room identity is 0. As the number of visited rooms grows from 2 to 139, every room is always handled by the same set of mechanism functions. Rooms differ in the environment data, not in separate mechanism implementations written for each room.

The Appendix I example mentioned by the reviewer (Appendix K in the revision) is a concrete instance of this structure: the ice mechanism is reused directly in a new room after about 25,000 actions and at least 20 context compactions. Before and after the agent enters that room, all 58 top-level functions and classes stay unchanged, and the only addition is the geometry data of the room. With this program, the agent plans a 38-step route with A*, and every transition during execution matches the program's prediction.

The audit shows that different rooms are handled by shared mechanism code, while room-specific information is kept in the geometry data. The ice case further shows what this structure does in practice: without modifying any mechanism function, the agent uses a learned rule to plan in a new room, and successfully executes a route in which every predicted step matches the observation. Together, the two pieces of evidence support mechanism reuse across rooms.

On interaction cost (Q7). We agree with the reviewer that a large number of exploratory actions does not mean poor abstraction. In the MazeBench run in the paper, out of 1,517 committed action plans, about 69% explicitly state in the plan description that the plan is meant to probe or verify a rule. Unexpected transitions that contradict the program's prediction account for only 3.7% of all actions.

So we do not infer the quality of the model's abstractions from action count or prediction failures alone. The new frozen-program experiment directly measures predictive transfer to unseen levels, and the program structure audit gives further evidence of shared rules and incremental knowledge accumulation.

The program structure audit and the interaction cost analysis above are now in Appendix N of the revision and are referenced in Sec 4.3.

2. Compositional generalization and hierarchical abstraction (W3, Q5)

The paper allows the agent to specify a goal predicate and search within its program (Section 3.3). But choosing a goal for a planning call is not the same as learning and reusing higher-level goal abstractions or hierarchical task structures.

If hierarchical goal abstraction is not a contribution of Schema, the text should say so clearly.

We agree that specifying a goal predicate at planning time is not the same as learning and reusing hierarchical goal abstractions. To clarify, learning hierarchical goal abstractions is not a research goal or contribution of this paper, and the paper does not make this claim. We do not propose a method for learning hierarchical options or reusable task decompositions.

The core contribution of Schema is to let the agent build, verify, and use an executable model of the environment on its own. Section 3.3 is about how the learned model is used for goal-directed planning: in each planning call, the agent chooses the target itself based on the current task and what it already knows. The target can be clearing the level, an intermediate waypoint, or a situation in which an uncertain mechanism can be tested. An intermediate waypoint here is just the target of a single planning call, not a subgoal abstraction that is learned and reused.

Mechanism reuse and compositional planning across levels (LS20). We added an analysis of LS20 in Appendix O of the revision. It lists the mechanisms present in each of the seven levels and uses program snapshots to show how mechanisms are accumulated and reused. In level 5, after updating its rule for the shape transformation T, the agent combines it with the existing rotation R as T→T→R→R to reach the target shape. The rotation function stays exactly the same in the program snapshots of levels 3–6, and when a moving button triggers a rotation, the program directly reuses this function. In level 6, the program goes further and treats shape transformation and rotation as two types of edges in the search graph, so that these operations are combined through search. This case shows concretely how learned mechanisms are reused in later levels and used together in planning.

AuthorsOpenReview ↗

Response to Reviewer 9AH1 [3/4]

3. Comparison with prior methods (W4, Q6)

The method is effective, but its main operations have substantial precedents. LLM-written world models, counterexample-based programme revision, model-based planning, and prediction monitoring are all established ideas. The paper's actual difference appears to be that a general-purpose LLM controls when and how to perform these operations through a common harness, including revisions to the state representation.

This is a legitimate architectural contribution. However, terms such as a new paradigm and the claim of being the first harness to achieve human-level mechanism discovery are rather strong.

A matched implementation of a prior model-learning-and-planning loop, or a more tightly controlled fixed-schedule variant using the same Schema tools, would help test the contribution of agent-directed control.

The contribution of Schema is to let the agent itself organize how an executable world model is learned and used. Based on feedback from interaction, the agent decides which hypotheses to propose, which experiments to design, when to check the history, and how to plan actions with its current model. It can also revise the mechanism rules and the state representation itself. What we call interactive program induction refers to exactly this design paradigm, in which program learning and action selection are both handed to the agent. Program synthesis, counterexample-based revision, and model-based planning all have important precedents. Our contribution is to let the agent organize these operations on its own based on the current evidence, so that it keeps building and using an executable understanding of the environment.

New comparison with WorldCoder. Following the reviewer's suggestion of a matched comparison with a prior model-learning-and-planning loop, we added a comparison between WorldCoder (Tang et al., 2024) and Schema on ARC-AGI-3. We adapted WorldCoder following Algorithm 1 of the original paper, keeping its REx refinement, Thompson sampling, optimistic model synthesis, and 5% random exploration. The experiment compares two designs: program learning controlled by a predefined procedure, and learning and interaction organized by the agent itself. Implementation details, per-game results, and analysis are in Appendix L of the revision (Table 16), and we added references to it in Sec 4.1 and Related Work.

To control for differences in interfaces and tool implementations, WorldCoder and Schema share the same program prediction interface, the same history consistency check (backtest), and the same BFS/A* planner. We made two adaptations to the original method: initial exploration tries each legal action once instead of taking 10 random actions, and the planner is the shared BFS/A* above instead of value iteration / MCTS.

We compare on the 10 hardest ARC-AGI-3 games used in the Table 1 ablations. Both methods use Claude Opus 4.8 (max effort), and each game gets 8 hours of effective wall-clock time, including LLM inference, tool execution, and planning. Schema's results are taken from the first 8 hours of the corresponding runs in Table 1.

Levels cleared / RHAE (%).

GameWorldCoderSchema
WA300 / 0.02 / 3.4
LF521 / 1.83 / 10.9
AR251 / 2.88 / 89.9
CN040 / 0.04 / 46.4
DC220 / 0.00 / 0.0
G50T0 / 0.04 / 35.7
M0R00 / 0.04 / 46.8
RE861 / 2.65 / 41.7
SK480 / 0.01 / 1.6
LS201 / 3.05 / 47.6
Total / mean4 / 1.036 / 32.4

We further looked into WorldCoder's reasoning traces. Two representative failure modes are:

  • In LF52, to fit the entire past history, the program creates a global list variable called "MEMORIZED" and then hardcodes all past counterexamples into it.

  • In DC22, the program has to satisfy the optimism requirement, i.e., some way to clear the level must be reachable. So after none of the 9 positions in the box triggered level completion, it attached extra conditions to these positions, such as "whether the pattern on the right has changed", and made these combinations new completion conditions only because they had not been tried yet.

A fixed synthesis-and-acceptance procedure easily leads the program to overfit the limited interaction record, meeting the requirements by memorizing counterexamples and piecing together completion conditions. Schema instead gives model revision, checking, planning, and environment interaction to the agent as tools, and lets it freely decide what to verify, which experiments to run, and when to update the model, so that it can discover the general mechanisms and the true completion conditions.

In addition, every WorldCoder revision requires regenerating the complete program. Repeated synthesis takes a lot of time and leaves fewer chances to collect new evidence from interaction.

AuthorsOpenReview ↗

Response to Reviewer 9AH1 [4/4]

4. Benchmark and evaluation (W5)

The ARC-AGI-3 results are impressive, but they are obtained on the public games. The paper's own Appendix C.1, Table 2, also lists GPT-6 Astra with a provider-adapter harness achieving 99.9% RHAE on the semi-private evaluation. This is a different set of games, so the figures are not directly comparable.

The paper should also report inference cost and tool-execution cost alongside environment actions.

Finally, the paper provides repeated runs on DiG-bench, while the long MazeBench result is essentially a trajectory-level case study. I would avoid making broad statistical claims about reliable long-horizon abstraction from that single trajectory.

On GPT-6 Astra: yes, as stated in App. C.1, the two results are not directly comparable. The key difference is the models' knowledge cutoff. ARC-AGI-3 was released in March 2026, and the three base models we use in the ARC-AGI-3 experiments all have knowledge cutoffs before that (January 2026 for Claude Opus 4.8 and Claude Fable 5, February 2026 for GPT-5.6 Sol). For evaluating mechanism discovery in unknown environments, this gives us a clear time separation from the benchmark release. In contrast, GPT-6 Astra has a knowledge cutoff of April 2026, which is after the release of ARC-AGI-3. Also, on the evaluation side, the GPT-6 Astra result is measured on the semi-private split (not open to academia), which is a different dataset from ours.

The gains of Schema are supported by direct comparisons with the same base model in the same evaluation environment.

  • On ARC-AGI-3, the RHAE of the same Claude Fable 5 rises from 58.7% in Claude Code to 99.2% in Schema (Figure 1a).

  • On MazeBench, at the same number of actions, the same GPT-6 Astra collects 30 gems and visits 133 rooms in Schema, but only 23 gems and 113 rooms in Codex (Figure 8).

This also shows that Schema brings a clear improvement even for GPT-6 Astra itself.

The benchmark splits, base models, harnesses, and scoring protocols are already explicitly documented in Figure 1, Section 4, and Appendix C.1. We now also clarify the benchmark split in the revised abstract.

For inference cost and tool-execution cost, we added a cost comparison on ARC-AGI-3 in Appendix D of the revision (Table 8). Across the four model configurations, Schema uses about the same number of billed tokens per game as the corresponding coding harness, or fewer, while achieving better task performance. The table also reports the share of time spent in tool calls, which covers execution overhead such as history backtesting and search-based planning.

In addition, on DiG-bench, Schema's token cost per cleared level is 3.40 USD, about 15% lower than the 4.00 USD of the basic harness (Figure 6). On MazeBench, at a token cost similar to Codex, Schema collects 7 more gems and visits 20 more rooms (Figure 8d).

We also ran an additional, independent full MazeBench run, with the same harness, base model (GPT-6 Astra), starting room, and observation setting as the main experiment. The new run collects 34 gems and visits 145 rooms within 26,850 actions, reproducing the long-horizon exploration performance of the main experiment. In both runs, the number of gems and the room coverage exceed the median of the top-50 human players.

RunActionsGemsRooms visited
Run in the paper27,81933139
New run26,85034145
Top-50 human median41,67529137

The results of this repeated run are now in Appendix G of the revision.

We also added a code analysis of MazeBench (Appendix N), in which we checked the program snapshots saved at every gem collection in both runs. In neither run do the mechanism functions branch on room identity: different rooms are handled by shared mechanism code, and the differences between rooms are kept in the geometry data. This provides program-level evidence that Schema learns and reuses general mechanisms during long-horizon exploration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

Dear Reviewers,

Thank you for your thoughtful feedback, which has been invaluable in helping us strengthen our paper. Below we summarize the main revisions and additional results.

  • Predictive generalization: Frozen-program evaluation on 500+ held-out levels achieves 63.1% changed-cell accuracy, versus 31.1% for the early-program control, without code revision (Appendix M).

  • Mechanism reuse: Program audits across 139 MazeBench rooms show shared mechanism functions with no room-identity branches. We also demonstrate cross-level mechanism reuse and compositional planning in LS20 (Appendices N, O).

  • Comparison with WorldCoder: Under the same base model and time budget, Schema clears 36 levels versus WorldCoder's 4 on 10 challenging ARC-AGI-3 games (Appendix L).

  • Executable vs. prose models: On 15 levels using the same learned rules, executable search clears all 15, while code-based and text-based reasoning each clear 8 (Appendix H).

  • Robustness and transparency: An independent MazeBench run achieves 34 gems and 145 rooms. We also added full prompts, baseline specifications, cost analyses, and clarified limitations (Appendices B, D, G).

We believe these additions strengthen the evidence for our contributions and address the reviewers' key concerns. We welcome any further questions or discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.