- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes WorldOcc, a novel real-time occupancy simulator for closed-loop autonomous driving evaluation. Unlike existing approaches based on learned world models, WorldOcc disentangles scene transitions from actor policies, enabling efficient and high-fidelity occupancy simulation. The authors further introduce two types of actor policies, namely normal and adversarial behaviors, to support diverse traffic interactions. Extensive experiments evaluate occupancy fidelity and efficiency, actor behaviors, and the performance of different planning methods. The proposed WorldOcc-Bench also provides a useful evaluation framework for occupancy-based planning.
Critical strengths
- Well-motivated and elegant design. The proposed policy-scene disentangled architecture provides a simple and effective solution to improving the fidelity and efficiency of occupancy simulation. The overall framework is technically sound and well organized.
- Comprehensive experimental evaluation. The authors conduct extensive experiments covering simulation fidelity, efficiency, actor behaviors, and closed-loop planning performance. The results demonstrate the effectiveness of the proposed framework.
Critical weaknesses
- Limited practical relevance of occupancy-based benchmarking. Although the proposed simulator is well designed, I still have concerns about the practical relevance of an occupancy-based planning benchmark. The evaluated planners rely on semantic occupancy representations rather than realistic camera or LiDAR observations. While occupancy is a meaningful intermediate representation, the benchmark does not explicitly account for perception errors and uncertainties that are unavoidable in real-world autonomous driving. Therefore, it remains unclear to what extent the benchmark results reflect the actual performance of planning methods in real-world deployment. I suggest the authors to clarify this limitation and further discuss the relationship between occupancy-based evaluation and real-world driving performance.
- Insufficient analysis of the realism of adversarial actor behaviors. Although Figure 4 and Appendix Figure 7 provide quantitative evaluations of actor behaviors, and Appendix Figure 8 presents a qualitative comparison, I believe further analysis is necessary to establish the realism of the learned adversarial policy. In particular, how do adversarial actors approach and interact with the ego vehicle? Do they exhibit realistic but aggressive driving behaviors, or do they simply move toward the ego vehicle in an unrealistic manner? Additional quantitative analysis of interaction patterns would help distinguish realistic adversarial driving from reckless behavior.
- Limited consideration of lane-level semantics and traffic rules. The benchmark considers drivable areas through semantic occupancy and related evaluation metrics. However, explicit lane-level map information and traffic rules appear to be insufficiently modeled. Lane boundaries, traffic lights, right-of-way, and traffic-rule compliance are important aspects of autonomous driving, particularly when evaluating interactions with aggressive traffic participants.
Questions for authors
None.
Dear Reviewer, I hope you are well. I wanted to kindly follow up on my earlier response and would be grateful for your thoughts when you have a moment.