Specification-Driven Generation and Evaluation of Discrete-Event World Models via the DEVS Formalism
Abstract
World models are central to LLM agents that must evaluate actions over long horizons. Yet much existing work focuses on environments governed by physical dynamics or spatial structure, whereas many high-impact domains, including supply chains, procurement networks, and business processes, evolve through discrete events, timing constraints, and causal dependencies. These settings call for discrete-event world models. Hand-engineered simulators provide consistency but are costly to build and adapt, while neural models can accumulate inconsistencies over long rollouts. We introduce DEVS-Gen, an LLM pipeline that generates discrete-event simulators from natural-language specifications using the Discrete Event System Specification (DEVS) formalism. It first plans the component hierarchy and interfaces, then synthesizes component behavior under these interface constraints. For evaluation, we develop a benchmark based on completed specifications that state the required interface and trace-observable behavior. We validate simulator traces against these operational and behavioral requirements, enabling automated scoring and rule-level diagnostics. When averaged across four backbones, DEVS-Gen achieves the highest operational and behavioral scores among the evaluated methods, although its relative performance varies by backbone. Its behavioral gains over both non-executing baselines are statistically significant, whereas those over the execution-enabled baselines are not. It also requires less generation time and fewer tokens to reach higher operational and behavioral performance than the baselines.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.