- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
They introduce RoleWorld: transferring world models within envs by describing functional roles of objects (rather than their identities). An LLM (teacher) learns roles, rules, transition roles and a skill about decision-making. In a different environment another LLM (actor) keeps the rules and updates the roles of each object (through feedback and probing). RoleWorld beats baselines on MARS and also is able to correct wrong roles and params.
Critical strengths
The paper is well written, clear and also the results are interesting.
Critical weaknesses
Are the functional roles really what drives the improvement? There is no baseline with object-specific rules and the same skill and adaptation (to understand what roles are actually adding). Also, I think RoleWorld’s rules and skill are also built with a stronger model than some baselines use, so part of the gain could come from that?
Questions for authors
- How would object-specific rules perform with the same skill and adaptation?
- What happens if a new environment requires a role that the model has not learned? Can it detect this?
- Would the corruption experiment give similar results if you started with the teacher-generated model instead of the validated one?
4 comments
Question 2: Unexplained-observation detection and replay analysis.
Yes, this can be detected. The case raised by the reviewer, where the target world requires a function or mechanism absent from the source world, manifests in RoleWorld as observations that cannot be explained by any evaluated hypothesis combination. As described in Sec. 3.2, RoleWorld marks such observations as unexplained and triggers hypothesis expansion or revision rather than forcing a probability update; a revision is accepted only if it improves the cumulative log feedback score over the current episode (Eq. 2). Since observations with a feedback score of 0 under both the current and the proposed combinations are excluded from Eq. 2, such observations cannot cause an incorrect revision to be accepted. Moreover, the actor always selects actions from the full action space based on raw observations and the interaction history, rather than relying solely on role predictions (Sec. 3.3). Therefore, even when encountering a mechanism absent from the source world, RoleWorld does not force it into its role estimates, and the actor can still make decisions based on actual feedback.
To illustrate this, we replay RoleWorld's interaction records in each target world and measure the fraction of unexplained interactions under the frozen initial role prior and under the role estimates after online adaptation (backbone: Qwen3-8B).
| Terrain | Survival | Task Dep. | Terr. Surv. | Terr. Task. | Surv. Task. | All Three | |
|---|---|---|---|---|---|---|---|
| Init | 3.3% | 3.9% | 8.9% | 16.6% | 21.5% | 8.4% | 23.1% |
| Final | 3.2% | 3.2% | 1.5% | 4.2% | 5.0% | 1.4% | 4.3% |
As shown in the table, a small fraction of interactions remain unexplained in each target world, yet RoleWorld still outperforms all baselines, indicating that these few unexplained observations neither corrupt role estimation nor hinder the actor's decisions.
Question 3: Choice of the validated prior and recovery from teacher estimates.
A teacher-generated model is not a suitable starting point for this controlled experiment, but the effect of online refinement from such a model is already reflected in our transfer experiments. Starting from the validated reference prior ensures that the injected corruption level matches the actual error rate and that recovery can be compared directly against the reference values. A teacher-generated model contains unknown errors of its own, which would be confounded with the injected corruption and make both the corruption level and the recovery difficult to measure accurately. In the transfer experiments, RoleWorld initializes each target world with the teacher-inferred role estimates, which do not fully match the target world. After online refinement, the fraction of unexplained interactions drops from as high as 23.1% to at most 5.0% (see our response to Q2). We have added this rationale for starting from the validated reference prior rather than a teacher-generated model to Sec. 4.2.
Thank you for the detailed response and the new experiments!! :)
The Obj-World comparison answers my main concern. Because it uses the same teacher, skill and adaptation procedure, it isolates the role-based representation, and roles still come out clearly ahead. The WorldCoder comparison under matched teacher and budget, together with the Obj-World result, also addresses my question about the teacher model. I understand that the IFR, PoE-World and WALL-E comparison asks a different question (transfer versus relearning in the target world).
On new mechanisms, the explanation of how unexplained observations are handled makes sense, and the replay analysis is helpful. Since the MARS variants stay within the role library by design, though, this shows that the method avoids corrupting its estimates rather than that it can deal with genuinely new dynamics.
My concerns have been addressed, and I will raise my soundness score!
Thanks a lot for the insightful discussion and for all the time you put into reviewing our work.
We thank the reviewer for the careful review and helpful comments, which we address point by point below.
Yes. Our controls indicate that the role-based representation contributes substantially to the gains. To examine this, we construct a variant, Obj-World, that replaces the role-based representation with an object-specific one: each object type has its own fields, and DSL transition rules are defined over object types rather than role prototypes. Obj-World is induced by the same teacher in the source world under the same source-learning budget. The actor, skill construction procedure, online adaptation procedure, and evaluation protocol are identical to those of RoleWorld; only the units they operate on change with the representation: online inference assigns each object to a source-world object type rather than to role prototypes, and the skill takes the corresponding object information as input in place of role information. The results are as follows:
The results show that RoleWorld outperforms the object-specific variant with both backbones, with the largest reward gap on Task Dep. Since Obj-World shares the teacher, actor, skill construction procedure, and online adaptation procedure with RoleWorld, this gap reflects the contribution of the role-based representation itself.
The gains do not stem from a stronger teacher. The comparison between RoleWorld and WorldCoder uses the same teacher LLM, source world, source-learning budget, and target-world backbone. Under these matched conditions, RoleWorld still achieves higher average reward and score than WorldCoder with both backbones (Table 2), indicating that the performance gap does not stem from the teacher. Moreover, Obj-World is induced by the same teacher yet still underperforms RoleWorld, which further supports this conclusion.