acceptodds
Under review as a conference paper at ICLR 2027

RoleWorld: Knowledge Transfer in World Modeling with Functional Role Abstractions

Abstract

World models enable large language model (LLM) agents to anticipate action effects and make informed decisions. For generalization, it is desirable for such world models to transfer efficiently to new environments with similar interaction structure and underlying dynamics. Recent approaches encode world dynamics as symbolic rules or executable programs, whose compositional structure in principle affords transfer, but still organize such knowledge around environment-specific objects. To address this limitation, we introduce *RoleWorld*, a symbolic world modeling approach which uses reusable prototypes and environment-specific parameter values as *functional role abstractions* to represent how objects behave in interactions. From interaction trajectories, a teacher LLM extracts role prototypes and executable transition rules defined over role instances, together with a skill that guides the actor LLM to use role information in decision making. In a target world, without the teacher, the actor uses this skill together with its current role assignments and parameter estimates, and actively tests a role instance hypothesis through interaction when its uncertainty affects the current decision. Feedback from each action refines role assignments and estimates parameter values online, with no LLM parameter updates and the transition rules held fixed. Experiments on MARS benchmark show that RoleWorld enables robust generalization across environments with significantly altered object behavior, suggesting that functional role abstractions can serve as an effective component for knowledge transfer in world modeling. Our code and data will be open-sourced upon formal release.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
symbolic world model, neuro-symbolic learning, transfer learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_hw4SOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

They introduce RoleWorld: transferring world models within envs by describing functional roles of objects (rather than their identities). An LLM (teacher) learns roles, rules, transition roles and a skill about decision-making. In a different environment another LLM (actor) keeps the rules and updates the roles of each object (through feedback and probing). RoleWorld beats baselines on MARS and also is able to correct wrong roles and params.

Critical strengths

The paper is well written, clear and also the results are interesting.

Critical weaknesses

Are the functional roles really what drives the improvement? There is no baseline with object-specific rules and the same skill and adaptation (to understand what roles are actually adding). Also, I think RoleWorld’s rules and skill are also built with a stronger model than some baselines use, so part of the gain could come from that?

Questions for authors
  1. How would object-specific rules perform with the same skill and adaptation?
  2. What happens if a new environment requires a role that the model has not learned? Can it detect this?
  3. Would the corruption experiment give similar results if you started with the teacher-generated model instead of the validated one?
4 comments

We thank the reviewer for the careful review and helpful comments, which we address point by point below.

Weakness 1 and Question 1: Object-specific ablation and same-teacher comparison.

Yes. Our controls indicate that the role-based representation contributes substantially to the gains. To examine this, we construct a variant, Obj-World, that replaces the role-based representation with an object-specific one: each object type has its own fields, and DSL transition rules are defined over object types rather than role prototypes. Obj-World is induced by the same teacher in the source world under the same source-learning budget. The actor, skill construction procedure, online adaptation procedure, and evaluation protocol are identical to those of RoleWorld; only the units they operate on change with the representation: online inference assigns each object to a source-world object type rather than to role prototypes, and the skill takes the corresponding object information as input in place of role information. The results are as follows:

MetricsMod. TypeQwen3-8BGPT-5.6 Luna
Obj-WorldOursObj-WorldOurs
RewardTerrain6.3 ± 2.69.4 ± 3.77.2 ± 3.811.4 ± 1.1
Survival6.0 ± 2.58.0 ± 2.66.8 ± 3.310.8 ± 2.4
Task Dep.2.3 ± 1.06.8 ± 0.83.2 ± 2.58.5 ± 1.5
Terr. Surv.5.7 ± 3.18.1 ± 3.37.5 ± 2.611.0 ± 1.7
Terr. Task.3.3 ± 0.96.2 ± 2.14.8 ± 1.87.0 ± 1.0
Surv. Task.2.4 ± 1.34.3 ± 0.83.5 ± 1.24.7 ± 0.8
All Three1.9 ± 1.53.0 ± 1.52.2 ± 1.13.3 ± 1.5
All4.06.55.08.1
Score (%)Terrain7.9 ± 3.416.6 ± 9.410.0 ± 0.219.2 ± 5.1
Survival7.0 ± 2.410.6 ± 5.610.2 ± 5.619.1 ± 7.4
Task Dep.1.2 ± 0.44.9 ± 1.32.8 ± 1.19.9 ± 4.8
Terr. Surv.5.6 ± 1.711.0 ± 4.89.4 ± 2.217.6 ± 7.6
Terr. Task.1.7 ± 0.45.8 ± 2.93.3 ± 1.55.6 ± 1.6
Surv. Task.1.7 ± 0.34.4 ± 1.72.0 ± 0.24.1 ± 2.5
All Three1.7 ± 0.62.2 ± 0.91.4 ± 0.22.4 ± 0.5
All3.87.95.611.1

The results show that RoleWorld outperforms the object-specific variant with both backbones, with the largest reward gap on Task Dep. Since Obj-World shares the teacher, actor, skill construction procedure, and online adaptation procedure with RoleWorld, this gap reflects the contribution of the role-based representation itself.

The gains do not stem from a stronger teacher. The comparison between RoleWorld and WorldCoder uses the same teacher LLM, source world, source-learning budget, and target-world backbone. Under these matched conditions, RoleWorld still achieves higher average reward and score than WorldCoder with both backbones (Table 2), indicating that the performance gap does not stem from the teacher. Moreover, Obj-World is induced by the same teacher yet still underperforms RoleWorld, which further supports this conclusion.

Question 2: Unexplained-observation detection and replay analysis.

Yes, this can be detected. The case raised by the reviewer, where the target world requires a function or mechanism absent from the source world, manifests in RoleWorld as observations that cannot be explained by any evaluated hypothesis combination. As described in Sec. 3.2, RoleWorld marks such observations as unexplained and triggers hypothesis expansion or revision rather than forcing a probability update; a revision is accepted only if it improves the cumulative log feedback score over the current episode (Eq. 2). Since observations with a feedback score of 0 under both the current and the proposed combinations are excluded from Eq. 2, such observations cannot cause an incorrect revision to be accepted. Moreover, the actor always selects actions from the full action space based on raw observations and the interaction history, rather than relying solely on role predictions (Sec. 3.3). Therefore, even when encountering a mechanism absent from the source world, RoleWorld does not force it into its role estimates, and the actor can still make decisions based on actual feedback.

To illustrate this, we replay RoleWorld's interaction records in each target world and measure the fraction of unexplained interactions under the frozen initial role prior and under the role estimates after online adaptation (backbone: Qwen3-8B).

TerrainSurvivalTask Dep.Terr. Surv.Terr. Task.Surv. Task.All Three
Init3.3%3.9%8.9%16.6%21.5%8.4%23.1%
Final3.2%3.2%1.5%4.2%5.0%1.4%4.3%

As shown in the table, a small fraction of interactions remain unexplained in each target world, yet RoleWorld still outperforms all baselines, indicating that these few unexplained observations neither corrupt role estimation nor hinder the actor's decisions.

Question 3: Choice of the validated prior and recovery from teacher estimates.

A teacher-generated model is not a suitable starting point for this controlled experiment, but the effect of online refinement from such a model is already reflected in our transfer experiments. Starting from the validated reference prior ensures that the injected corruption level matches the actual error rate and that recovery can be compared directly against the reference values. A teacher-generated model contains unknown errors of its own, which would be confounded with the injected corruption and make both the corruption level and the recovery difficult to measure accurately. In the transfer experiments, RoleWorld initializes each target world with the teacher-inferred role estimates, which do not fully match the target world. After online refinement, the fraction of unexplained interactions drops from as high as 23.1% to at most 5.0% (see our response to Q2). We have added this rationale for starting from the validated reference prior rather than a teacher-generated model to Sec. 4.2.

Thank you for the detailed response and the new experiments!! :)

The Obj-World comparison answers my main concern. Because it uses the same teacher, skill and adaptation procedure, it isolates the role-based representation, and roles still come out clearly ahead. The WorldCoder comparison under matched teacher and budget, together with the Obj-World result, also addresses my question about the teacher model. I understand that the IFR, PoE-World and WALL-E comparison asks a different question (transfer versus relearning in the target world).

On new mechanisms, the explanation of how unexplained observations are handled makes sense, and the replay analysis is helpful. Since the MARS variants stay within the role library by design, though, this shows that the method avoids corrupting its estimates rather than that it can deal with genuinely new dynamics.

My concerns have been addressed, and I will raise my soundness score!

Thanks a lot for the insightful discussion and for all the time you put into reviewing our work.

Reviewer_e4onOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RoleWorld, a symbolic world modeling method for LLM agents that transfers knowledge between relevant environments through functional role abstractions composed of reusable role prototypes and environment-specific parameter values. Teacher LLM derives role prototypes, actionable transition rules based on role instances expressed by predefined DSLs, and skill programs that organize role information for the actor from source world trajectories and controlled interactions. In the target world, the prototypes, rules, and skills are fixed. The actor maintains probabilistic role instance hypotheses initialized with source priors and updates them using exact-match feedback scores. Parameters are modified before reconsidering role assignments, modifications are adopted only if they improve the cumulative log feedback score, and hypotheses are actively tested when uncertainty influences the decision.

Assessments of seven variants of Crafter-based MARS showed that RoleWorld, transitioned from Default, was the highest in both average reward and score. As a result of ablation, removing the skill reduces the reward to the level of Direct LLM, with online adaptation and active verification each contributing, and a hand-designed symbolic planner using the same representation achieves much higher performance.

Critical strengths
  • A design that separates reusable role prototypes and executable rules from environment-specific object-role assignments and parameters, and employs parameter-first modification order and cumulative evidence-based adoption criteria, directly targets failure scenarios where familiar objects perform different functions depending on the environment.
  • Through careful verification of the representations themselves beyond task rewards, we demonstrate that role-based models inherently contain useful transferable knowledge.
Critical weaknesses
  • Performance improvement relies heavily on the skills generated by the teacher rather than the role abstraction itself. Even if only the skills are removed while presenting the action requirements and predictive effects derived from the role as they are, the reward drops from 6.5 to 1.2, becoming similar to Direct LLM (1.0) and lower than WorldCoder (2.5); however, this skill is only briefly explained in the text, and no baseline receives a corresponding decision-support layer. Therefore, the superiority of the main table cannot be attributed to the effect of the functional role abstraction itself, and a comparison is needed, for example, by placing the same decision-support program on top of the WorldCoder model.

  • The comparison mixes the strength of the teacher model with privileged access in the source world. While the RoleWorld world model and skills are constructed using environment interaction tools that allow for state restoration and setting control conditions in GPT-5.6 Sol, IFR, PoE-World, and WALL-E learn directly from the target backbone, and it is unclear whether these controlled interactions are included in the 5-episode source budget shared with WorldCoder. Furthermore, since the event vocabulary, operators, interpreters, and execution semantics of the predefined DSL provide significant domain structure, this transition comparison is difficult to regard as a comparison between methods with similar prior knowledge.

  • The scope of the evaluation is directly aligned with the assumptions of the method. The paper deals only with target world variations expressible within the transferred library; since the MARS variant maintains the Crafter's objects and achievements, target worlds where fixed prototypes and rules fail due to the need for functions or mechanisms not present in the source are not tested, nor are fallbacks for such cases explained. Control tests are also limited to the default traversal, harvesting, crafting, and placement, excluding NPC and survival dynamics, which are behaviors modified by the MARS variant.

  • The contribution of online adaptation is uneven. With source priors fixed, w/o adaptations were slightly higher than the full model in 3 out of 7 variants, and the average improvement due to adaptation mostly stems from task depth. Even in the GPT-5.6 Luna environment, differences compared to IFR or WALL-E across various variants are within one standard deviation, suggesting that the evidence that online role refinement drives transfer is weaker than what the average figures imply.

Questions for authors
  • Can you describe the skill program in detail (its size, which parts are teacher-generated versus predefined, and what it computes) and report a baseline that adds a comparable decision-support layer to WorldCoder or another world-model baseline?
  • Are the teacher’s environment-interaction tools (state restoration and controlled conditions) counted within the five-episode source budget, and does WorldCoder have access to equivalent tools?
  • Which parts of online refinement are performed by the backbone LLM and which are algorithmic, and how often are observations unexplained in practice?
8 comments

We are grateful to the reviewer for the careful reading of our paper and the valuable feedback, which has helped us substantially improve the quality and clarity of this work.

Weakness 1 and Question 1: Skill details and controlled comparisons with WorldCoder + Skill and Obj-World. (part 1)

The skill consists of a program (about 200 lines of Python) and a prompt template (about 2,400 characters). Its inputs are the current role assignments and their probabilities, the DSL predictions for candidate actions, the current state, and the interaction history. It adds no environment knowledge beyond the roles and the DSL; it only decides which information is kept and which is emphasized. At each step it runs three stages:

  1. Filtering (predefined): For each action, keep only the DSL predictions under the highest-probability combination of role instance hypotheses it depends on, and drop outcomes with zero probability in the current state.

  2. Teacher-written hints (teacher-generated): In the source world, the teacher observed what role information the actor tended to miss or misuse, and wrote two programs that organize it: (i) compare the current state with rule conditions and state missing prerequisites as hints; (ii) highlight actions predicted to reduce health. Both programs only change how role information is presented; they do not make decisions for the actor. Once generated, the skill stays fixed in target worlds, and the teacher never accesses any target world.

  3. Merging (predefined): Append the role information, the filtered DSL predictions (with probabilities and verification status), and the hints to the original decision prompt, together with two active-verification prompts: (i) verify unverified roles; (ii) try actions whose conditions the DSL predicts are currently satisfied.

We have expanded and consolidated the implementation details of the skill in Appendix A.3 (w/o Skill).

This also explains the reward drop in w/o Skill. That setting keeps only a fixed basic filter, so predictions are not further narrowed by the current state. Replaying recorded trajectories shows that role information then adds about 5,000 tokens to each decision prompt on average, more than the actor can use effectively. The skill reduces this to about 500 tokens relevant to the current state, so the actor can use the information and performance improves.

Weakness 1 and Question 1: Skill details and controlled comparisons with WorldCoder + Skill and Obj-World. (part 2)

Separating the contributions of the skill and the role abstraction. The skill organizes information derived from the role representation, so it cannot be applied directly to methods without roles. To separate the contribution of the skill from that of the role abstraction itself, we add two experiments:

  1. WorldCoder + Skill. Following the reviewer's suggestion, we add a functionally comparable decision-support layer to WorldCoder by turning the parts of the skill that apply to WorldCoder into search rules in its MCTS: the prerequisite check becomes pruning at node expansion, removing actions that its code world model predicts to be infeasible; the health-risk check becomes a penalty at node evaluation, lowering the scores of actions predicted to reduce health. World model learning, online code refinement, and the MCTS simulation budget remain unchanged.

  2. Obj-World. We construct a variant, Obj-World, that replaces the role-based representation with an object-specific one: each object type has its own fields, and the DSL transition rules are defined over object types rather than role prototypes. Obj-World is induced by the same teacher in the source world with the same budget. The actor, skill construction procedure, online adaptation procedure, and evaluation protocol are identical to those of RoleWorld; only the units they operate on change with the representation: online inference assigns each object to a source-world object type rather than to role prototypes, and the skill takes the corresponding object information as input in place of role information. The results are as follows:

MetricsMod. TypeQwen3-8BGPT-5.6 Luna
WorldCoderWorldCoder + SkillObj-WorldOursWorldCoderWorldCoder + SkillObj-WorldOurs
RewardTerrain5.3 ± 1.75.1 ± 1.86.3 ± 2.69.4 ± 3.75.8 ± 1.75.4 ± 1.67.2 ± 3.811.4 ± 1.1
Survival2.3 ± 1.43.2 ± 2.16.0 ± 2.58.0 ± 2.63.1 ± 1.73.8 ± 2.16.8 ± 3.310.8 ± 2.4
Task Dep.0.8 ± 0.80.8 ± 0.72.3 ± 1.06.8 ± 0.81.2 ± 0.61.0 ± 0.33.2 ± 2.58.5 ± 1.5
Terr. Surv.4.4 ± 1.64.7 ± 1.45.7 ± 3.18.1 ± 3.34.3 ± 1.44.8 ± 1.47.5 ± 2.611.0 ± 1.7
Terr. Task.3.7 ± 1.54.9 ± 0.83.3 ± 0.96.2 ± 2.14.2 ± 1.45.1 ± 0.84.8 ± 1.87.0 ± 1.0
Surv. Task.1.3 ± 0.60.4 ± 0.72.4 ± 1.34.3 ± 0.80.7 ± 0.80.5 ± 0.73.5 ± 1.24.7 ± 0.8
All Three-0.0 ± 0.6-0.2 ± 0.51.9 ± 1.53.0 ± 1.5-0.4 ± 0.5-0.1 ± 0.42.2 ± 1.13.3 ± 1.5
All2.52.74.06.52.72.95.08.1
Score (%)Terrain4.3 ± 1.24.4 ± 1.07.9 ± 3.416.6 ± 9.45.5 ± 0.54.6 ± 0.910.0 ± 0.219.2 ± 5.1
Survival1.5 ± 0.32.7 ± 0.37.0 ± 2.410.6 ± 5.62.3 ± 0.42.9 ± 0.210.2 ± 5.619.1 ± 7.4
Task Dep.0.6 ± 0.20.5 ± 0.01.2 ± 0.44.9 ± 1.30.7 ± 0.10.5 ± 0.02.8 ± 1.19.9 ± 4.8
Terr. Surv.3.4 ± 1.02.9 ± 1.05.6 ± 1.711.0 ± 4.82.8 ± 1.12.9 ± 1.09.4 ± 2.217.6 ± 7.6
Terr. Task.2.5 ± 0.12.8 ± 0.31.7 ± 0.45.8 ± 2.92.7 ± 0.33.1 ± 0.53.3 ± 1.55.6 ± 1.6
Surv. Task.1.0 ± 0.30.5 ± 0.01.7 ± 0.34.4 ± 1.70.6 ± 0.10.5 ± 0.02.0 ± 0.24.1 ± 2.5
All Three0.3 ± 0.10.2 ± 0.01.7 ± 0.62.2 ± 0.90.2 ± 0.10.2 ± 0.01.4 ± 0.22.4 ± 0.5
All1.92.03.87.92.12.15.611.1

WorldCoder + Skill performs on par with WorldCoder overall, while RoleWorld outperforms WorldCoder + Skill in all seven worlds. This is expected: the skill introduces no new knowledge and only organizes role information for the LLM actor, so it brings little benefit to methods without roles. The comparison with Obj-World shows that RoleWorld outperforms the object-specific model with both backbones, with the largest reward gap on Task Dep. Together, these results indicate that the gains cannot be attributed to the skill alone: with the same skill, object types perform worse than roles. The role-based representation thus contributes substantially, while the skill is needed to make role information usable by the actor.

Weakness 2 and Question 2: Comparison design, tool access for WorldCoder, budget accounting, and DSL priors. (part1)

The differences in learning conditions between RoleWorld and the baselines are deliberate: we designed each of the two comparisons to answer its own question (Table 1; Sec. 4, Evaluation protocol).

RoleWorld vs. WorldCoder. This comparison examines the effect of the knowledge representation. Both methods therefore use the same LLM for source learning (GPT-5.6 Sol), the same source world, the same source-learning budget, and the same target backbone. In target worlds, both use only the backbone, with no access to the source-learning LLM or any tools. The original WorldCoder has no tools for restoring states or setting controlled conditions, so in Table 2 we evaluate it as originally designed, without adding such tools. To rule out the possibility that the performance gap comes from tool access, we add a controlled experiment that gives WorldCoder the same environment-interaction tools as RoleWorld, attached at the same point: the step in source learning in Default where the model code is checked and revised against observed state transitions. All other settings are unchanged. The results are as follows:

MetricsMod. TypeQwen3-8BGPT-5.6 Luna
WorldCoderWorldCoder + ToolOursWorldCoderWorldCoder + ToolOurs
RewardTerrain5.3 ± 1.75.4 ± 1.99.4 ± 3.75.8 ± 1.75.3 ± 1.711.4 ± 1.1
Survival2.3 ± 1.43.4 ± 1.98.0 ± 2.63.1 ± 1.73.5 ± 2.210.8 ± 2.4
Task Dep.0.8 ± 0.81.0 ± 0.36.8 ± 0.81.2 ± 0.61.2 ± 0.38.5 ± 1.5
Terr. Surv.4.4 ± 1.65.4 ± 3.08.1 ± 3.34.3 ± 1.43.8 ± 2.411.0 ± 1.7
Terr. Task.3.7 ± 1.54.2 ± 1.46.2 ± 2.14.2 ± 1.44.0 ± 1.57.0 ± 1.0
Surv. Task.1.3 ± 0.61.0 ± 1.04.3 ± 0.80.7 ± 0.81.5 ± 1.84.7 ± 0.8
All Three-0.0 ± 0.6-0.3 ± 0.53.0 ± 1.5-0.4 ± 0.5-0.2 ± 0.53.3 ± 1.5
All2.52.96.52.72.78.1
Score (%)Terrain4.3 ± 1.25.1 ± 0.416.6 ± 9.45.5 ± 0.54.9 ± 1.219.2 ± 5.1
Survival1.5 ± 0.33.3 ± 1.610.6 ± 5.62.3 ± 0.43.0 ± 1.319.1 ± 7.4
Task Dep.0.6 ± 0.20.6 ± 0.14.9 ± 1.30.7 ± 0.10.6 ± 0.19.9 ± 4.8
Terr. Surv.3.4 ± 1.04.9 ± 2.311.0 ± 4.82.8 ± 1.12.7 ± 1.617.6 ± 7.6
Terr. Task.2.5 ± 0.12.8 ± 0.25.8 ± 2.92.7 ± 0.32.4 ± 0.35.6 ± 1.6
Surv. Task.1.0 ± 0.30.7 ± 0.14.4 ± 1.70.6 ± 0.11.5 ± 1.14.1 ± 2.5
All Three0.3 ± 0.10.2 ± 0.22.2 ± 0.90.2 ± 0.10.2 ± 0.02.4 ± 0.5
All1.92.57.92.12.211.1

With the tools added, WorldCoder improves slightly with Qwen3-8B and remains essentially unchanged with GPT-5.6 Luna, and it still trails RoleWorld with both backbones. This indicates that the performance gap stems mainly from what the two methods transfer, not from tool access. We have added these results to Sec. 4.1 and Appendix A.8.

Reviewer_KfgpOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

RoleWorld transfers symbolic world knowledge by separating reusable functional roles and transition rules from environment-specific object assignments and parameters. A teacher constructs the shared model and an actor skill in a source world. In target worlds, feedback updates role hypotheses while the shared rules and skill remain fixed, and the actor can test uncertain hypotheses. The paper gives an executable modeling and inference framework rather than a theoretical guarantee. MARS experiments evaluate transfer across altered dynamics, component ablations and a symbolic decision policy. Controlled corruption experiments additionally test recovery of role estimates and direct action predictions.

Critical strengths
  1. The paper presents a coherent transfer setting and states its scope clearly. Sec. 2 explicitly restricts target changes to those expressible within the transferred library, and Tabs. 1 and 2 distinguish source transfer from learning directly in a target world.

  2. The factorization in Sec. 3.1 provides a useful design principle. Objects can instantiate multiple compatible functions, while shared transition rules read instance-specific requirements and effects. This makes reuse possible without requiring familiar object names to retain their original behavior.

  3. The inference and decision components work toward the same objective. Eqs. 1 and 2 connect feedback to revisable role hypotheses, while the skill in Sec. 3.3 turns those hypotheses into action requirements and opportunities for informative interaction.

  4. The experiments examine the complete agent and several consequential design choices. Tab. 2 tests different types of environmental change with different backbones, and Tab. 3 probes adaptation, verification, decision support and the choice of decision policy. The paper also discusses where adaptation adds little and where the LLM actor remains a bottleneck.

  5. The controlled study goes beyond measuring task reward. Fig. 2 evaluates both symbolic recovery and held-out action predictions. App. A.6 checks the reference prior against executed transitions and injected errors, while distinguishing that controlled evidence from its more limited natural-trajectory diagnostic.

Critical weaknesses
  1. Tab. 3 does not isolate the contribution of role reassignment as cleanly as the interpretation in Sec. 4.1 suggests. Those signals guide the actor's verification decisions in Sec. 3.3. Its difference from the full method therefore combines a restriction on reassignment with a change to the information driving exploration.

  2. The operational details of the central inference procedure remain incomplete. After Eq. 1, the paper restricts updates to a finite set of high-probability hypothesis combinations without specifying the selection rule, retained set size or probability initialization after expansion. These choices determine whether a corrected binding can enter the search and retain enough probability to be tested. Without them, the reported recovery cannot be independently connected to a fully specified algorithm.

Questions for authors
  1. For Tab. 1 and the controlled interactions described in Sec. 3.1, did WorldCoder and RoleWorld receive the same state-restoration and intervention tools during source learning?

  2. How is the acceptance test in Eq. 2 implemented when an accumulated observation has zero feedback score under both the current and proposed hypotheses?

  3. Do any existing comparisons hold the actor's task organization and prerequisite guidance fixed while varying role-based versus object-specific transition models?

5 comments

We sincerely thank the reviewer for the time and effort devoted to reviewing our manuscript, as well as for the insightful comments and constructive suggestions.

Weakness 1: Role reassignment and exploration.

The purpose of exploration is exactly to verify the correctness of the corresponding role, at two levels: its role type and its parameters. Therefore, role and exploration are paired at each level: given one specific role, there is a specific exploration process. Fixed Role Assignments freezes only the role-type level, so we cannot isolate role reassignment from its corresponding role-type exploration; parameter-level exploration is retained, as interaction evidence still updates parameter values and hypothesis weights within the assigned prototypes. We have revised the wording in Sec. 4.1 and App. A.3 accordingly, and now attribute the gap between Fixed Role Assignments and the full model to role-type-level adaptation rather than to reassignment alone.

Weakness 2: Operational details.

Selection rule. The prior of a combination is the product of the probabilities of its constituent hypotheses. At each step, we select the 128 combinations with the highest prior and update them with Eq. 1. A hypothesis that appears in none of these combinations has its probability multiplied by their mean feedback score, so it is neither eliminated nor favored merely because it was not evaluated.

Size of the retained set. Each object retains at most 128 hypotheses, and at most 128 combinations are evaluated per step. A continuous parameter takes five equally spaced values over its prior range. Values outside these points can still enter through revision (see point 4).

Initialization after expansion. When a parameter field is expanded for the first time, the probability of the parent hypothesis is divided evenly among its child hypotheses. The resulting set is then truncated to 128 hypotheses and renormalized. Truncated values remain within the prior range and can re-enter through revision.

Can a corrected binding enter the search with sufficient probability? Yes, candidate values for a revision (Sec. 3.2) are drawn from observed feedback and from the original prior range. For list-valued fields such as prerequisites, the prior is halved for each additional item, which discourages overly complex explanations. A revised hypothesis does not inherit the suppressed probability of the hypothesis it replaces. Instead, its cumulative feedback score over the full history of the current episode is recomputed following Eq. 2 and combined with the complexity prior above. Its probability therefore reflects how well it explains the full history, not a value suppressed by earlier mispredictions. This allows a corrected binding to enter the evaluated combinations and be tested.

We have added these clarifications to Sec. 3.2 and Appendix A.7.

Question 1: Tool access in the WorldCoder comparison.

The original WorldCoder method does not include state-restoration or intervention tools, so we evaluated it in its original form without adding them. To rule out the possibility that the performance gap comes from differences in tool access, we ran an additional control experiment. In it, WorldCoder is given exactly the same tools as RoleWorld, connected at the same point: the source-learning step in Default world where model code is checked and revised against observed state transitions. All other settings are unchanged. The results are as follows:

MetricsMod. TypeQwen3-8BGPT-5.6 Luna
WorldCoderWorldCoder + ToolOursWorldCoderWorldCoder + ToolOurs
RewardTerrain5.3 ± 1.75.4 ± 1.99.4 ± 3.75.8 ± 1.75.3 ± 1.711.4 ± 1.1
Survival2.3 ± 1.43.4 ± 1.98.0 ± 2.63.1 ± 1.73.5 ± 2.210.8 ± 2.4
Task Dep.0.8 ± 0.81.0 ± 0.36.8 ± 0.81.2 ± 0.61.2 ± 0.38.5 ± 1.5
Terr. Surv.4.4 ± 1.65.4 ± 3.08.1 ± 3.34.3 ± 1.43.8 ± 2.411.0 ± 1.7
Terr. Task.3.7 ± 1.54.2 ± 1.46.2 ± 2.14.2 ± 1.44.0 ± 1.57.0 ± 1.0
Surv. Task.1.3 ± 0.61.0 ± 1.04.3 ± 0.80.7 ± 0.81.5 ± 1.84.7 ± 0.8
All Three-0.0 ± 0.6-0.3 ± 0.53.0 ± 1.5-0.4 ± 0.5-0.2 ± 0.53.3 ± 1.5
All2.52.96.52.72.78.1
Score (%)Terrain4.3 ± 1.25.1 ± 0.416.6 ± 9.45.5 ± 0.54.9 ± 1.219.2 ± 5.1
Survival1.5 ± 0.33.3 ± 1.610.6 ± 5.62.3 ± 0.43.0 ± 1.319.1 ± 7.4
Task Dep.0.6 ± 0.20.6 ± 0.14.9 ± 1.30.7 ± 0.10.6 ± 0.19.9 ± 4.8
Terr. Surv.3.4 ± 1.04.9 ± 2.311.0 ± 4.82.8 ± 1.12.7 ± 1.617.6 ± 7.6
Terr. Task.2.5 ± 0.12.8 ± 0.25.8 ± 2.92.7 ± 0.32.4 ± 0.35.6 ± 1.6
Surv. Task.1.0 ± 0.30.7 ± 0.14.4 ± 1.70.6 ± 0.11.5 ± 1.14.1 ± 2.5
All Three0.3 ± 0.10.2 ± 0.22.2 ± 0.90.2 ± 0.10.2 ± 0.02.4 ± 0.5
All1.92.57.92.12.211.1

With these tools, WorldCoder improves slightly on Qwen3-8B and is essentially unchanged on GPT-5.6 Luna. In both cases it still trails RoleWorld. This indicates that the performance gap comes mainly from what the two methods transfer, not from tool access. We have added these results to Sec. 4.1 and Appendix A.8.

Question 2: Zero-score observations in Eq. 2.

If an observation in the history receives a feedback score of 0 under both the current hypothesis and the proposed hypothesis, this indicates that the observation does not discriminate between the two hypotheses. It is therefore excluded from both sums in Eq. 2. We have added this clarification to Sec. 3.2.

Question 3: Role-based vs. object-specific transition models under a fixed skill.

In RoleWorld, both task organization and prerequisite guidance are provided by the skill. Following the reviewer’s suggestion, we constructed a variant, Obj-World, that replaces the role-based representation with an object-specific one. Each object type has its own fields, and DSL transition rules are defined over object types rather than role prototypes. Obj-World is induced by the same teacher in the source world with the same budget. The actor, skill construction procedure, online adaptation procedure, and evaluation protocol are identical to those of RoleWorld; only the units they operate on change with the representation: online inference assigns each object to a source-world object type rather than to role prototypes, and the skill takes the corresponding object information as input in place of role information. The results are as follows:

MetricsMod. TypeQwen3-8BGPT-5.6 Luna
Obj-WorldOursObj-WorldOurs
RewardTerrain6.3 ± 2.69.4 ± 3.77.2 ± 3.811.4 ± 1.1
Survival6.0 ± 2.58.0 ± 2.66.8 ± 3.310.8 ± 2.4
Task Dep.2.3 ± 1.06.8 ± 0.83.2 ± 2.58.5 ± 1.5
Terr. Surv.5.7 ± 3.18.1 ± 3.37.5 ± 2.611.0 ± 1.7
Terr. Task.3.3 ± 0.96.2 ± 2.14.8 ± 1.87.0 ± 1.0
Surv. Task.2.4 ± 1.34.3 ± 0.83.5 ± 1.24.7 ± 0.8
All Three1.9 ± 1.53.0 ± 1.52.2 ± 1.13.3 ± 1.5
All4.06.55.08.1
Score (%)Terrain7.9 ± 3.416.6 ± 9.410.0 ± 0.219.2 ± 5.1
Survival7.0 ± 2.410.6 ± 5.610.2 ± 5.619.1 ± 7.4
Task Dep.1.2 ± 0.44.9 ± 1.32.8 ± 1.19.9 ± 4.8
Terr. Surv.5.6 ± 1.711.0 ± 4.89.4 ± 2.217.6 ± 7.6
Terr. Task.1.7 ± 0.45.8 ± 2.93.3 ± 1.55.6 ± 1.6
Surv. Task.1.7 ± 0.34.4 ± 1.72.0 ± 0.24.1 ± 2.5
All Three1.7 ± 0.62.2 ± 0.91.4 ± 0.22.4 ± 0.5
All3.87.95.611.1

The results show that the role-based transition model outperforms the object-specific transition model on both backbones, with the largest reward gap on Task Dep. RoleWorld’s advantage therefore cannot be attributed solely to the skill’s task organization and prerequisite guidance; the role-based representation itself contributes substantially.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.