acceptodds
Withdrawn from ICLR 2027

OPEN WORLDS NEED RULE DISTILLATION

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Open-world AI systems are rapidly expanding through larger content spaces, richer task sets, more coherent agent behaviors, and more flexible language interfaces. This position paper argues that such progress masks a more fundamental bottleneck: systems may appear open because agents can see, say, or attempt more, while remaining closed at the mechanism layer that determines what the world can recognize, validate, execute, and reuse. We call this mismatch the content trap. We argue that the next bottleneck of open-world AI is not generating more content, tasks, or behaviors, but enabling the endogenous growth of executable world mechanisms. To this end, we propose rule distillation: treating LLM-generated intents, justifications, and consequence predictions not as direct world updates, but as candidate rule priors that must earn validity through mechanistic formalization, execution, replay validation, audit tracing, and failure rollback before becoming reusable world mechanisms. Through a scarcity-driven trajectory in the Farm-Town simulator, we illustrate how one-off generated behavior can be distilled into executable, verifiable, and reusable mechanisms. We therefore argue that the core progress of open worlds is not to generate more content, but to gradually acquire the ability to simulate real-world rules.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
open-world AI, rule distillation, large language models, multi-agent simulation, executable world mechanisms, open-ended environments

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_195gOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper argues, that current open-world AI systems primarily expand content, tasks, and agent behaviors without expanding the rules that govern what agents can actually do. To address this, the authors propose rule distillation, a framework in which LLM-generated proposals are converted into executable world mechanisms. Through experiments in a multi-agent simulation called Farm-Town, they demonstrate that agents can acquire new enforceable rules, while also showing that validation failures and limited generalization remain substantial challenges.

Critical strengths
  • open-world systems are an interesting research topic.
Critical weaknesses
  • In the very first sentence of the introduction, the authors state "Open-world systems face a persistent tension: agents should act in ways their designers did not anticipate, and those actions should reshape the world through mechanisms that accept them and produce traceable consequence". The authors do not seem to define what an open-world system means in the context of their paper at this point.
  • It remains somewhat unclear what the authors mean by "rules" and "hypothesis".
  • The link between open-worlds in the setting the paper seems to describe and open endedness in line 158-159 requires further explanation.
  • A related work section seems to be missing from the main body.
  • Overall the paper is very hard to parse and the overall takeaway of the work is unclear to me
  • The design choices for Farm Town are not explained.
  • The captions of the figures are too short and make the figures difficult to interpret. For example, Figure 2 shows a "temporal agent graph", without describing what is actually shown in the Figure. Other instances, such as Figure 5 show overlapping text inside the figure, that is not readable.
Questions for authors
  • What is the main application the authors have in mind when talking about open-world systems?
Reviewer_wYUpOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper has a central thesis that one important part of the definition of "open-endedness" is that open worlds must introduce new rule systems, not just new content.

Critical strengths
  1. The underlying hypothesis itself is interesting, and with proper implementation could be useful.
Critical weaknesses
  1. Overall, the paper was nearly incomprehensible. For instance, "Farm-town" was suddenly introduced with little to no justification despite being central to the paper. Different versions of the environment like V1-V10 were introduced without explanation, etc.

  2. The paper read a bit like a position paper, saying that something is necessary without really demonstrating why it is necessary. There was not a coherent philosophical or empirical argument made for why it was necessary.

Questions for authors

None

Reviewer_bi6bOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The submission presents itself as a position paper, arguing that open-world agent learning is currently limited by generation of more world content, but lacks generation of more mechanisms. It then proposes a framework to overcome this limitation, rule distillation, based on LLM outputs, and tests it in the setting of a recourse-constrained game simulator, Farm-Town.

Critical strengths
  • The topic open-world agent learning is relevant and timely.
  • The presented methodology covers state-of-the-art LLMs.
  • The submission provides extensive details of its analysis.
Critical weaknesses
  • The manuscript writing is too unclear for publication. It relies on technical jargon and assumptions about specific forms of open-word learning without clearly stating or justifying them.
  • The submission is not actually a position paper. Instead of stating a position on a machine learning topic, it uses the claimed shortcomings to propose a new method, which it then tests empirically. Yet, for a scientific contributions, the presented empirical evaluation is insufficient, lacking diversity of tasks and baseline methods.
  • The presented material is not sufficiently curated. With 77 pages, the manuscript is simply too long for a human reader to properly benefit from all material. The authors should reduce the presented material to the aspects relevant to understanding and judging the work, moving the remaining details e.g. in a code repository.
Questions for authors

I do not have any questions that could affect my assessment. The manuscript would need a major revision to be suitable for a top-tier conference such as ICLR. This is not possible in a rebuttal format.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.