acceptodds
Under review as a conference paper at ICLR 2027

WorldOcc: A Real-Time Occupancy Simulator Towards Closed-Loop Evaluation with Diverse and Learnable Traffic Behaviors

Abstract

Closed-loop simulation for autonomous driving has garnered widespread attention in recent years, yet, to our knowledge, no occupancy simulator is at once real-time, high-fidelity, long-horizon, and able to exercise diverse, learnable traffic. This shortfall stems largely from the prevailing practice of building occupancy simulators on a learned world model, which entangles scene evolution with the decision making of traffic participants (i.e., actors). We therefore rethink whether an occupancy simulator must rely on a learned world model, and propose WorldOcc, an occupancy simulator that is fast, faithful, and sustained over long rollouts, without a learned world model. At its core is a policy-scene disentangled architecture that models the scene transition and the actor policy independently, so each serves its own end, the transition for fidelity and the policy for diversity and flexibility. For high fidelity over a long horizon, we design a lossless rigid-operator transition, Ego Warp-Recenter and Actor Erase-Print, that advances the occupancy state with bounded errors. For diverse traffic behaviors, we tailor a plug-and-play actor policy network that learns and deploys behaviors, from normal to adversarial, effectively inside the simulator. WorldOcc simulates at about 1270 FPS on an RTX 4090 GPU while holding IoU ≥ 70% over 16 s. Beyond simulation, we introduce WorldOcc-Bench, to our knowledge the first closed-loop benchmark for occupancy planners with diverse traffic, benchmarking six recent planner families. WorldOcc not only reveals the vulnerability of occupancy planners under closed-loop conditions with diverse traffic, but also shows potential as an effective post-training environment.

Primary area:
applications to robotics, autonomy, planning
Keywords:
semantic occupancy, closed-loop simulator, benchmarking, autonomous driving.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qY9POpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes WorldOcc, a novel real-time occupancy simulator for closed-loop autonomous driving evaluation. Unlike existing approaches based on learned world models, WorldOcc disentangles scene transitions from actor policies, enabling efficient and high-fidelity occupancy simulation. The authors further introduce two types of actor policies, namely normal and adversarial behaviors, to support diverse traffic interactions. Extensive experiments evaluate occupancy fidelity and efficiency, actor behaviors, and the performance of different planning methods. The proposed WorldOcc-Bench also provides a useful evaluation framework for occupancy-based planning.

Critical strengths
  1. Well-motivated and elegant design. The proposed policy-scene disentangled architecture provides a simple and effective solution to improving the fidelity and efficiency of occupancy simulation. The overall framework is technically sound and well organized.
  2. Comprehensive experimental evaluation. The authors conduct extensive experiments covering simulation fidelity, efficiency, actor behaviors, and closed-loop planning performance. The results demonstrate the effectiveness of the proposed framework.
Critical weaknesses
  1. Limited practical relevance of occupancy-based benchmarking. Although the proposed simulator is well designed, I still have concerns about the practical relevance of an occupancy-based planning benchmark. The evaluated planners rely on semantic occupancy representations rather than realistic camera or LiDAR observations. While occupancy is a meaningful intermediate representation, the benchmark does not explicitly account for perception errors and uncertainties that are unavoidable in real-world autonomous driving. Therefore, it remains unclear to what extent the benchmark results reflect the actual performance of planning methods in real-world deployment. I suggest the authors to clarify this limitation and further discuss the relationship between occupancy-based evaluation and real-world driving performance.
  2. Insufficient analysis of the realism of adversarial actor behaviors. Although Figure 4 and Appendix Figure 7 provide quantitative evaluations of actor behaviors, and Appendix Figure 8 presents a qualitative comparison, I believe further analysis is necessary to establish the realism of the learned adversarial policy. In particular, how do adversarial actors approach and interact with the ego vehicle? Do they exhibit realistic but aggressive driving behaviors, or do they simply move toward the ego vehicle in an unrealistic manner? Additional quantitative analysis of interaction patterns would help distinguish realistic adversarial driving from reckless behavior.
  3. Limited consideration of lane-level semantics and traffic rules. The benchmark considers drivable areas through semantic occupancy and related evaluation metrics. However, explicit lane-level map information and traffic rules appear to be insufficiently modeled. Lane boundaries, traffic lights, right-of-way, and traffic-rule compliance are important aspects of autonomous driving, particularly when evaluating interactions with aggressive traffic participants.
Questions for authors

None.

Reviewer_UPETOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes WorldOcc, a closed-loop traffic simulator in the semantic-occupancy modality that deliberately avoids a learned world model. Prior occupancy world models couple scene evolution and actor behavior into a single forward model. which the authors argue causes three limitations: low per-step fidelity, short rollouts, and non-reactive actor behavior tied to the recording. WorldOcc introduces a policy-scene disentangled architecture with two components:

  1. a rigid-operator transition (Ego Warp-Recenter + Actor Erase- Print) that advances occupancy by anchoring to the next recorded reference state, with a bounded, non-accumulating per-step error.
  2. a plug-and-play actor-policy network trained inside the simulator; behavior cloning for normal traffic and PPO against an ego-centric adversarial reward for adversarial traffic.

The authors also release WorldOcc-Bench, the first closed-loop occupancy-planner benchmark, with six planner families evaluated under three traffic settings (recorded, normal, adversarial). The benchmark exposes substantial degradation of current occupancy-based planners under adversarial traffic, and preliminary results suggest the simulator is also useful as a post-training environment via Reinforcement Learning Fine-Tuning (RLFT).

Critical strengths
  • clearly demonstrated. Table 2 shows a large fidelity gap. Even in the per-step teacher-forcing setting of Table 8, the most charitable regime for learned baselines, WorldOcc still leads by 17-27 points.
  • Clean conceptual contribution. Decoupling the transition function where fidelity matters from the actor policy where diversity matters is a well-motivated factorization of the closed- loop simulation problem. The POMDP formulation cleanly separates these components and makes the design easy to compose with external planners.
  • Non-trivial technical details. Ego Warp-Recenter and Actor Erase-Print are conceptually simple but engineered carefully: Actor Erase-Print preserves class and shape via static capture-and-stamp (Appendix A.4), and the per-step error bound (Eq. 19) does not accumulate because each step re-derives occupancy from the reference state rather than from the previous prediction.
  • Real-time speed is a differentiator. 1270 FPS on GPU and 436 FPS on CPU make the simulator practical as a post-training environment, which the authors demonstrate preliminarily with the WorldEngine RLFT recipe.
  • Benchmark effort is substantial. Six planner families are reproduced; the normal/ adversarial traffic axis is well structured; Figure 5's planner-by-traffic matrix is informative; and the interesting finding that even the GT-Following oracle drops from 0.864 to 0.753 under adversarial traffic.
  • Appendices are thorough. The error bound (A.3), heading / translation deviation sensitivity (Tables 10, 11), operator timing (B.10), kinematic models (B.6-B.7), and substep calibration (B.2) are all documented.
Critical weaknesses
  • "Lossless" overstates what is proven. The abstract and the main text use "lossless rigid-operator transition," while Appendix A.3 derives bounded, non-accumulating error (Eq. 19) with concrete finite bounds. I would recommend "bounded-error" or "non-accumulating-error" throughout the main text, or at minimum a clearer caveat in the abstract.
  • Scope limitation: not a scene generator. WorldOcc re-drives recorded scenes rather than synthesizing novel ones. This is a legitimate design choice, but it means the "diverse traffic" axis of WorldOcc-Bench comes from re-driving existing actors, not from novel scenes, weather, or geometry. Comparisons to true generative world models are therefore partly apples-to-oranges on the generative axis, even though WorldOcc clearly wins on fidelity.
  • Fidelity is sensitive to heading deviation. Table 10b shows IoU drops from 1.00 at 0° to 0.49 at 1° and 0.19 at 8° of heading error. The reported 73-80% IoU holds because benchmarked planners stay close to the recorded route (ADE 0.32-0.63 m). A policy that intentionally deviates — exactly the kind one would want to stress- test — would see smaller gains. This dependency is quantified in the appendix but deserves one line in the main text (§4.3) so readers do not over-generalize the fidelity claim.
  • Comparisons with learned world models require careful framing. DOME, OccWorld, I²-World, and DynamicCity are trained to predict future scenes from a start state, not to re-drive against a reference. The paper acknowledges this (Appendix B.3), but Table 2's headline numbers may read as a direct capability comparison. A short caveat in the Table 2 caption would help.
  • Adversarial actor policy is relatively narrow. The reward (Eq. 14) is essentially "collide with ego" with a sensing-range penalty. A richer taxonomy of safety-critical behaviors (cut-in, hard brake, red-light running — listed in future work) is not implemented. The WorldOcc-Score "adversarial" column therefore reflects a specific adversarial regime rather than the full space of safety-critical interactions.
Questions for authors
  • Can WorldOcc be composed with an occupancy scene generator at initialization to extend coverage beyond recorded scenes? What is the fidelity cost of such composition?
  • For the six benchmarked planners, which were used as released vs. adapted to the occupancy modality? Please add a one-line summary per planner (modality bridging method, if any).
  • The adversarial actor is trained to collide. Have you measured realism of its generated behavior against real safety-critical datasets (CARLA Leaderboard, DeepAccident), or does the reward permit physically plausible but behaviorally unrealistic attacks?
  • The RLFT result is promising but limited to WorldEngine. Does it generalize across the benchmarked planner families? Table 4 does not currently include RLFT entries for others.
  • How expensive is training the actor-policy network relative to training a learned occupancy world model? If the actor policy is substantially cheaper, this is an additional argument for the disentangled design and worth headlining.
1 comment

Dear Reviewer, I hope you are well. I wanted to kindly follow up on my earlier response and would be grateful for your thoughts when you have a moment.

Reviewer_nnm7OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

WorldOcc is an occupancy driving simulator without a learned world model: it warps the recorded next frame to the ego’s driven pose and re-stamps actors moved by a learned policy (normal or adversarial). It claims IoU ≥ 70% over 16 s at about 1270 FPS, beating learned world models on Occ3D-nuScenes and Occ3D-Waymo.

They also introduce WorldOcc-Bench, a closed loop benchmark of occupancy planners under recorded, normal and adversarial traffic.

Critical strengths

A simple, useful engineering insight. Anchoring each step to the recorded next frame and only re-centring and re-stamping gives a fast, stable occupancy loop.

Critical weaknesses

The footnote says “Code and webpage are available” but in the repo there’s no code.

WorldOcc reads the ground-truth next frame s′_{t+1} at every step. That makes it log-replay pseudo-simulation [1] plus warping, not a simulator in the same class as the world models.

Fairer baselines would be other replay and reactive simulators [1–6].

[7] is cited only as a scene generator, but it is itself a long-horizon closed-loop occupancy simulator. That weakens both the “first” claim and the “about 3 s horizon” framing.

References

[1] D. Dauner et al. NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking. NeurIPS 2024 (Datasets and Benchmarks Track). arXiv:2406.15349.

[2] N. Karnchanachari et al. Towards Learning-Based Planning: The nuPlan Benchmark for Real-World Autonomous Driving. ICRA 2024.

[3] C. Gulino et al. Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research. NeurIPS 2023 (Datasets and Benchmarks Track). arXiv:2310.08710.

[4] S. Kazemkhani, A. Pandya, D. Cornelisse, B. Shacklett, E. Vinitsky. GPUDrive: Data-Driven, Multi-Agent Driving Simulation at 1 Million FPS. ICLR 2025. arXiv:2408.01584.

[5] Q. Li, Z. Peng, L. Feng, Q. Zhang, Z. Xue, B. Zhou. MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning. IEEE TPAMI, 2023. DOI: 10.1109/TPAMI.2022.3190471. arXiv:2109.12674.

[6] Q. Li, Z. Peng, L. Feng, Z. Liu, C. Duan, W. Mo, B. Zhou. ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modeling. NeurIPS 2023 (Datasets and Benchmarks Track). arXiv:2306.12241.

[7] T. Liu, S. Zhao, M. Pourkeshavarz, W. Li, N. Rhinehart. OccSim: Multi-Kilometer Simulation with Long-Horizon Occupancy World Models. arXiv:2603.28887, 2026.

Questions for authors

Can you please provide explanation for the following number mismatches?

  1. Table 2: the Avg column says 75.3/74.0, but the mean of the per-horizon values is 75.1/73.1. Table 8 lists the same per-horizon values with Avg 75.1/73.1.

  2. Table 7 vs Table 3: Table 7 gives Waymo, K=2, 16 s ground-truth replay as 84.2/80.3. Table 3 gives 92.84/91.65 for what looks like the same setting.

Ego-MLP’s adversarial score is 0.065 at every k, including k=1. Go-Straight, which also uses no perception, only drops from 0.224 to 0.221. Is this an input mismatch or a bug?

1 comment

Thank you for the detailed response. Q1 (Avg convention) is well adressed.

The original ResWorld entry listed ten specific authors and ICLR 2025. I could find no paper by that author group. Occ-LLM listed a non-existent first author with the correct title and venue. These do not seem like “incomplete drafting-time placeholders” to me as authors claim. I note the revised bibliography is correct and leave the assessment to the AC.

Concerns

  1. Table 11 reports IoU = 1.000 at zero pose deviation. Under recorded ego actions, Tables 2–3 therefore measure tracking error against the log, not scene fidelity. The baselines forecast unseen futures from recorded history, open-loop; this is not "closed-loop fidelity at 1–3 s". That [1–6] do not output occupancy is true but does not address this point.

  2. The submitted caption and body text both specified a 16 s horizon; the new description (30 scenes, ~20 s clips) cannot be verified. Since the paper claims non-accumulating error (Table 3 is flat at 92.6–93.5%), it is unclear how about 4 extra seconds or a 30-scene subset would cost 8.6 IoU points.

  3. The saturation explanation is not supported by the paper's own data. GT-Following also ignores other agents, yet its adversarial score falls steadily with k (0.810 → 0.627, temporal), while Ego-MLP stays at 0.065 from k=1. With 143/150 episodes terminated at k=1, it is unclear why additional adversaries never shorten an episode. The figures cited (RC 0.113, 67 collisions, 76 deviations) do not appear in the revised paper, and Appendix D.7 still describes Ego-MLP as falling 0.24 → 0.06 with k, which Table 19 does not show.

Requested Clarifications

  1. Qualify the "high-fidelity, IoU ≥ 70%" claim in the abstract and Table 2, or add a plain occupancy log-replay baseline (recorded frames, no warping) to Table 2, so readers can see how much of the IoU comes from the recorded frame itself.

  2. Report the K sweep under the Table 3 protocol (202 scenes, 16 s); at about 1270 FPS this costs seconds per K. Please also reconcile the step count: 27,492 steps over 202 scenes is about 136 steps (13.6 s) per scene, not 16 s at 10 Hz.

  3. Report per-k collision counts and mean termination step for GT-Following, Ego-MLP and Go-Straight under adversarial traffic, add them to the paper, and correct Appendix D.7 to match Table 19.

  4. Table 3: Avg equals the 16 s column exactly in all four WorldOcc entries, unlike Table 2. Please state what each column measures.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.