- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
experience replay, and complex reward design. The authors propose DRL Plus, in which a shared policy network outputs steering, throttle, and braking commands. A calculator predicts the vehicle’s next position from these actions, constructs a dynamic target rectangle, and computes a geometric loss. A verification module checks the candidate action and continues optimizing the policy if the result is not accepted. A finite-state machine identifies the current driving phase so that the shared network can be trained with the corresponding phase-specific loss. The paper’s main contribution is the integration of geometric learning signals and online action verification into a control pipeline that does not train a critic, perform Bellman backups, or maintain an experience replay buffer. For validation, the authors derive a zero failure probability under assumptions including invariance of the safe-state set; compare DRL Plus with DDPG, SAC, CPO, and PPO-Lagrangian in CARLA using phase completion rates, overall success rates, and control metrics, reporting a 100% overall success rate across 1,500 episodes; and use PRISM to check success, collision, and violation properties in an abstract overtaking model.
Critical strengths
A concrete alternative to value-based policy learning. DRL Plus combines predicted next-state geometry, a task-specific loss, and online action verification without a critic, Bellman backups, or experience replay. This is consequential because it offers a testable design for structured, safety-critical control tasks that goes beyond modifying a conventional reward function.
Critical weaknesses
-
The training objectives are not clearly connected. The paper does not explain how the objective in Eq. (12) on page 5, the geometric loss returned by Algorithm 1 on page 7, the (J(θ)) update in Algorithm 2, and the stage-specific losses on page 16 form a single executable optimization procedure. The verification rule and its reliability are unclear. The acceptance criterion used by Verify is not sufficiently specified, and the paper does not analyze how errors in the predicted next position affect its decisions. Thepremise of the safety guarantee is not established. Equations (8)–(11) assume that the selected actions remain safe under the true transition dynamics, but the paper does not show that actions accepted by the verifier satisfy this condition.
-
The contribution of the integration is not clearly established. The rectangular regions, finite-state machine, and stage-specific losses involve substantial task-specific design. The paper should clarify what this particular combination adds beyond existing prediction and safety-filtering methods. Targeted comparisons are missing. The current baselines do not isolate whether the reported improvement comes from the learning rule, geometric regions, stage decomposition, or online action filtering. Controlled comparisons with appropriate alternatives are needed. The current experiments do not adequately support the paper’s broader claims. The reported results from 1,500 episodes in one CARLA overtaking scenario do not establish safety across scenarios, a general solution to long-horizon credit assignment, or computational efficiency at decision time.
3.On page 7, the item labeled “Figure 4” is a numerical table. The text also refers to this success-rate table as “Table 4,” although Table 4 on page 8 reports the statistical test results. Please correct the table label and numbering, and update the corresponding in-text references.
Questions for authors
Please specify the training objective and parameter-update rule actually used. How do the safety-probability estimate and loss in Eq. (12) on page 5 relate to the geometric loss in Algorithm 1 on page 7, the (J(θ)) update in Algorithm 2, and the three stage-specific losses on page 16? Do gradients propagate through the next-position predictor? What is the exact acceptance criterion used by Verify? Given prediction errors and the motion of other vehicles, why must an accepted action satisfy the true-transition safety condition required by Eq. (8)? How does the PRISM abstraction correspond to the policy executed in CARLA? Were the 1,500 episodes used to calculate the success rates in Figure 4 run independently after training with the policy frozen? How do they relate to the 1,500 training episodes listed in Table 2? What are the success rates for each of the 25 random seeds, and what was the unit of analysis in the statistical tests? What controlled results distinguish the effects of the learning rule, dynamic regions, finite-state machine, and online verification on the success rate? Compared with predictive-control or safety-filtering methods given comparable task-specific information and decision-time computational budgets, what incremental benefit does the proposed integration provide?
Objective clarification, expanded experiments, and corrected presentation
We thank the reviewer for the detailed comments on mathematical consistency, evaluation, and presentation. The revision clarifies the methodology text, expands the experiments, and corrects the success-table labeling.
Critical Weakness 1 and questions on the objective and gradients. Section 3.1 now defines the geometric loss between the predicted next position and the zone reference point in equation (6). Equation (7) combines it with the implemented regularization terms. The text states that the policy update uses ∇θLt through the differentiable calculator, with gradient clipping. Appendix A.2 distinguishes this local objective from the terminal performance measure JH(Π). The earlier description of three phase-specific losses has been removed: maneuver phases determine the zone and reference point. Locations: Section 3.1, equations (6)–(7), lines 250–258, p. 5, and lines 270–274, p. 6; Appendix A.2, lines 754–755, p. 14.
Critical Weakness 1 and questions on verification. Figure 2 now shows the predicted-position zone check, retention of feasible candidates, stopping decisions, and fallback selection when no feasible candidate is cached. The analytical discussion separately specifies uncertainty-set containment for the conditional safety argument. It explains why checking only the nominal predicted position is insufficient near a zone boundary and why position containment alone does not establish collision avoidance. Locations: Figure 2 and lines 180–193, p. 4; Section 3, lines 213–246, pp. 4–5; Section 3.1, lines 266–269, p. 5.
Critical Weakness 2: Formulation and broader evaluation. The revised discussion distinguishes refinement of policy-network parameters from direct optimization of control variables. We also added MPC to overtaking and introduced a U-turn scenario with all six algorithms. DRL Plus and MPC both achieve 100% overall overtaking success. In U-turns, DRL Plus achieves 100% observed success compared with 6.80% for the evaluated MPC configuration. The revised conclusion summarizes the complete framework's performance across these two maneuvers. Locations: Section 3.1, lines 259–274, pp. 5–6; Section 4.1, lines 282–295, p. 6; Tables 2–3, p. 8; Section 6, lines 497–509, p. 10.
Critical Weakness 3: Table labeling and references. The overtaking success-rate display is now Table 2, with its in-text reference corrected. Statistical significance results are presented separately in Table 6. Figure 4 now identifies the driving-scene illustration at simulation step 73. Locations: Figure 4 and lines 366–368, p. 7; Table 2, lines 378–389, p. 8; statistical-table reference, lines 421–424, p. 8; Table 6, p. 9.
Questions on seed aggregation and statistical notation. The revised experimental text clarifies that Tables 2–3 aggregate results across 25 random seeds. Appendix A.3 defines the stage-performance value for run/seed i and identifies N as the number of runs. The run/seed is therefore the stated unit in the statistical formulation. Locations: Section 4.1, lines 362–364, p. 7; Appendix A.3, lines 783–809, p. 15; randomization settings in Table 9, p. 16.
Question on PRISM correspondence. The new Appendix A.7 explains how controller action selection affects model interpretation and distinguishes abstract-model probabilities from deployed-controller performance. It identifies the abstraction map, event definitions, transition construction, and independent trajectory comparisons required to validate correspondence with CARLA. Locations: Appendix A.7, lines 949–959, p. 18, and lines 983–1025, p. 19.