- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces JaxNaviSim, a GPU-based maritime navigation simulator and benchmark built from real AIS records. It supports traffic replay and single- and multi-vessel control, and compares imitation learning and reinforcement learning methods. It also introduces PAR, which checks predicted collisions and adjusts proposed actions. Experiments across eight regions examine navigation performance, collision rates, and different dynamics settings.
Critical strengths
1: The platform addresses a useful problem. Combining real vessel trajectories with parallel simulation provides a useful resource for studying navigation in dense maritime traffic.
2: The experiments cover several learning approaches and settings. The comparisons show that better goal reaching does not necessarily mean fewer collisions. PAR substantially reduces collisions in the reported experiments while largely maintaining goal-reaching performance.
Critical weaknesses
1: The test set is used for model and parameter selection. Checkpoints and the PAR configuration are selected using test-set results. This makes the final evaluation less independent and may produce optimistic results. A separate validation set and an untouched test set are needed to establish the reliability of the reported gains.
2: PAR’s safety benefits and technical contribution need stronger support. The main results use simplified position and heading updates and interpolated AIS states. It is unclear whether the gains hold under stricter vessel dynamics or delayed, missing, and noisy observations. PAR also combines fairly standard ideas: constant-velocity prediction, collision checking, and candidate action selection. Comparisons with established safety-filtering methods would help establish what its specific design adds.
Questions for authors
1: How do the results change when checkpoints and PAR parameters are selected on a separate validation set and evaluated on an untouched test set? 2: Do PAR’s collision reductions hold under the alternative dynamics models, and when surrounding-vessel observations are delayed, missing, or inaccurate? 3: How does PAR compare with established collision-avoidance or safety-filtering methods under the same observations and motion constraints? 4: The main text relies heavily on text and tables, while the appendix diagrams are too schematic to explain the key mechanisms. Please add a clearer framework figure showing how AIS data become simulation scenarios, how policies control vessels, and where PAR intervenes. A concrete avoidance example showing the proposed action, alternatives, and final choice would also help.
I keep the original rate. The matched replay benchmark addresses my timing concern. My main concern remains unresolved because QC-FQL was trained with FM-BC teachers selected on the original Oct–Dec test split, and reselecting the saved QC-FQL checkpoints on Oct cannot undo the training-stage dependency. PAR also retains C3, which was chosen using the AEW and AESPA test populations, meaning the re-analysis still falls short of an independent Nov–Dec comparison for these results.