- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper argues, that current open-world AI systems primarily expand content, tasks, and agent behaviors without expanding the rules that govern what agents can actually do. To address this, the authors propose rule distillation, a framework in which LLM-generated proposals are converted into executable world mechanisms. Through experiments in a multi-agent simulation called Farm-Town, they demonstrate that agents can acquire new enforceable rules, while also showing that validation failures and limited generalization remain substantial challenges.
Critical strengths
- open-world systems are an interesting research topic.
Critical weaknesses
- In the very first sentence of the introduction, the authors state "Open-world systems face a persistent tension: agents should act in ways their designers did not anticipate, and those actions should reshape the world through mechanisms that accept them and produce traceable consequence". The authors do not seem to define what an open-world system means in the context of their paper at this point.
- It remains somewhat unclear what the authors mean by "rules" and "hypothesis".
- The link between open-worlds in the setting the paper seems to describe and open endedness in line 158-159 requires further explanation.
- A related work section seems to be missing from the main body.
- Overall the paper is very hard to parse and the overall takeaway of the work is unclear to me
- The design choices for Farm Town are not explained.
- The captions of the figures are too short and make the figures difficult to interpret. For example, Figure 2 shows a "temporal agent graph", without describing what is actually shown in the Figure. Other instances, such as Figure 5 show overlapping text inside the figure, that is not readable.
Questions for authors
- What is the main application the authors have in mind when talking about open-world systems?