- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces CabinSafeBench to evaluate the safety adaptation of cockpit agents under changing driving contexts. It constructs paired intervention scenarios by keeping user requests and task goals fixed while changing safety-relevant contexts, and develops CockpitWorld, a closed-loop simulation environment that couples Android Automotive OS (AAOS) with CARLA. The authors evaluate six LLM-based cockpit agents using metrics including Task Success Rate (TSR), Safe Task Success Rate (STSR), and Context Adaptation Failure Rate (CAFR). The results show that task success remains high after context interventions, while safe task success drops substantially.
Critical strengths
-
Practical research problem. The paper highlights the distinction between task completion and safe execution. By controlling context changes, it studies agents' safety adaptation and offers a reasonable perspective on safety evaluation beyond conventional task success rates.
-
Executable closed-loop evaluation framework. CockpitWorld couples AAOS with CARLA and uses deterministic, machine-checkable rules to evaluate task execution. This allows action safety to be assessed under specific environmental states rather than relying solely on natural-language outputs.
Critical weaknesses
-
Limited benchmark size, model coverage, and interpretability of aggregate results. The core intervention experiment contains only 90 paired scenarios, which may not adequately cover the diversity of cockpit safety challenges. The six models are also not compared under controlled model-scale settings, limiting the representativeness and generalizability of the findings. In addition, Table 1 shows dramatic differences in STSR, TSG, and CAFR across models released around the same time. The paper does not sufficiently discuss or explain these differences, nor does it provide error-analysis experiments to establish whether they stem from model capabilities rather than confounding factors such as inference configurations, tool-call integration, or the evaluation pipeline. The paper also does not explain why six models from different families form a representative model set. Averaging these heterogeneous models lacks a clearly defined statistical population, making the resulting scores difficult to interpret as indicators of broader cockpit-agent capabilities or specific capability gaps.
-
Insufficient verification of goal preservation, safety ground truth, and simulation validity. Keeping the task goal unchanged is a core design principle, but the natural-language request and the underlying goal predicate may be ambiguous. The current evidence does not establish that alternative executions in all intervention pairs satisfy the same task goal, nor does it allow independent verification that safe feasibility is strictly guaranteed in both contexts. Moreover, deterministic, machine-checkable evaluations ensure consistent rule execution but do not establish that the rules themselves are valid. The paper provides insufficient justification for the sources, triggering conditions, and key thresholds of its safety constraints, leaving it unclear whether STSR and CAFR accurately reflect actual safety adaptation capabilities. The generation of dynamic driving contexts, state updates, and specific scenario configurations also remain insufficiently clear. The limit of four interactions per episode makes the coverage of long-horizon tasks difficult to assess, limiting independent verification of the simulation capabilities and experimental results.
Figure-related issues. The Dense Traffic example in Figure 1 does not clearly show dense traffic. Figure 4 contains overlapping elements within its subplots, reducing readability.
Questions for authors
-
Benchmark size and coverage: Are 90 paired context-intervention scenarios sufficient? What supports the representativeness of these scenarios and the generalizability of the findings?
-
Cross-model differences and evaluation reliability: How can the extreme cross-model performance differences be explained or shown to reflect genuine model capabilities rather than evaluation artifacts? Why do the actual inference configurations differ across models?
-
Interpretation of Macro Avg: What is the statistical meaning of averaging results across six different model families? How should this metric be interpreted without a clear justification for the representativeness of the selected models?
-
Goal preservation: When the literal user request may differ from the underlying task goal, how do the authors verify that an alternative execution after intervention still satisfies the original goal? Can specific pairs illustrate how task goals are defined and safe feasibility is verified?
-
Safety ground truth: What are the sources, operational definitions, triggering conditions, and key thresholds of the safety constraints? How is the validity of these rules independently verified to ensure that the evaluation results are not biased by the rules themselves?
4 comments
Update Review
Thanks for detailed response. However, several key concerns remain insufficiently addressed.
1. Benchmark Size and Representativeness.
The paper still lacks sufficient analysis of coverage across task types, context combinations, and scenario complexity. Could the authors provide a sample-size convergence analysis or evaluation of additional tasks and intervention scenarios to demonstrate that the current benchmark size is sufficient to support the main conclusions?
2. Unexpected Cross-Model Performance and Evaluation Reliability.
The authors emphasize that all models use the same tool interface and basic inference settings, but this does not adequately explain the extreme performance differences, particularly the unexpected gaps between stronger and weaker models. The common-eligibility analysis only rules out differences in CAFR denominators; it does not establish that the results reflect genuine model capabilities rather than differences in actual inference configurations, tool integration, or scoring rules. As a benchmark, the paper should provide concrete failure analyses or controlled experiments to explain these results, which deviate substantially from typical capability expectations, rather than merely reporting score differences.
3. Interpretability of Macro Avg.
I understand that the authors define Macro Avg. as a descriptive mean across the six evaluated models rather than a population-level estimate. However, it remains unclear why these six models were selected and whether they are representative in terms of widespread use or state-of-the-art capabilities. Without a clear justification for model selection, the average is highly dependent on the chosen model set, limiting its value as a main aggregate result and potentially making it misleading.
We thank the reviewer for the follow-up comments. We provide additional quantitative evidence from our existing experimental data.
1. Benchmark Size and Representativeness.
CabinSafeBench contains 220 episodes per agent, including 40 baseline episodes and 90 aligned intervention pairs. We further conducted a sample-size stability analysis with 2,000 repeated subsamples, preserving pair alignment across models. At 15 pairs, the estimated ΔSTSR is 59.61 pp (95% interval: [53.33, 65.56]), compared with 59.63 pp over all 90 pairs; CAFR is 59.91% versus 59.97%. These results demonstrate consistent estimates of intervention-induced safety degradation across sample sizes within the evaluated suite. We additionally analyzed task distributions, context combinations, and interaction complexity to characterize the benchmark's operational coverage.
2. Cross-Model Performance and Evaluation Reliability.
Matched execution records directly reveal distinct action-selection behaviors across models. We aligned all 90 intervention pairs across four models and compared their actions under identical task and context specifications. In representative high-speed, heavy-rain, and closed-road scenarios, GPT and DeepSeek execute context-inappropriate target actions, whereas Gemini and Grok withhold the corresponding writes. Furthermore, our audit of five available direct-execution event streams confirms that all 1,083 executor-submitted tool calls had schema-valid recorded arguments and successful tool responses. These execution-level observations demonstrate genuine differences in action selection, independently observable before aggregate scoring. The analysis therefore provides concrete behavioral evidence for the pronounced cross-model differences.
3. Macro Average and Model Selection.
The six models were selected to provide cross-provider and cross-family coverage, enabling comparison of heterogeneous LLM agents under a unified evaluation protocol. Macro Avg. represents their equally weighted descriptive mean. We further conducted a leave-one-model-out sensitivity analysis, obtaining STSR values of 70.91–78.64% and CAFR values of 52.18–70.85%. We report these results alongside individual model scores and paired intervention analyses to characterize model-specific behavior and aggregate sensitivity. The primary contribution is a controlled evaluation framework that exposes context-adaptive safety failures through goal-preserving interventions.
Official Comment by Reviewer ZMBU
Thanks for response. Unfortunately, the authors still have not provided sufficient analysis of scenario coverage or detailed error analysis. Although they have added a subsampling stability experiment and execution-log audits for some models, this evidence is still insufficient to establish the representativeness of the current dataset or adequately explain the extreme performance differences across models. For example, DeepSeek V4.1 Flash has a CAFR of 98.89%, while Gemini 3.8 Flash has only 5.56%, a difference of 93.33 percentage points. Many similar discrepancies can be found in the main table. If such substantial score differences are primarily driven by a subset of specific scenarios or particular action-selection patterns, this further raises my concerns about the adequacy of scenario coverage and the validity of the evaluation design.
Furthermore, the authors still have not provided sufficient evidence or references to justify the representativeness of the six selected models. Although the authors define Macro Avg as a descriptive average, the paper continues to rely heavily on these averages throughout its analysis and conclusions. Without a clear justification for model selection, these results are highly susceptible to being overinterpreted as reflecting the overall capabilities of the field, which could be misleading.
In conclusion, I believe these core concerns remain unresolved. I remain seriously concerned about the evaluation validity of this benchmark and the reliability of its conclusions, and therefore maintain my initial assessment and rating.
Benchmark Validity and Evaluation Reliability
We thank the reviewer for recognizing the importance of our research problem and executable evaluation framework. We clarify the experimental design and evaluation validity below.
W1 & Q1/Q3: Benchmark Coverage and Statistical Interpretation.
CabinSafeBench contains 220 episodes per agent, comprising 40 baseline episodes and 90 nominal–intervened pairs, spanning six task families and six safety-relevant context dimensions. The 90 pairs constitute the canonical controlled-intervention set, specifically designed to measure safety adaptation under fixed task objectives. Importantly, our conclusions are supported by paired statistical analysis rather than aggregate scores alone. As reported in Appendix D, aligned pair-cluster bootstrap resampling (10,000 replicates) yields a 59.63 percentage-point STSR decrease (95% CI: [56.67, 62.41]), while TSR remains essentially unchanged. The Macro Avg. represents the equally weighted descriptive mean across the six evaluated agents, not a population-level estimate of all cockpit agents. We additionally report individual model results and context-specific profiles to explicitly characterize heterogeneity rather than obscure it through aggregation.
W1 & Q2: Cross-Model Differences and Inference Reliability.
The six agents are evaluated on identical benchmark instances using the same system-prompt template, observation format, 20 semantic tools, and interaction budget. As detailed in Appendix B, all models use the same OpenAI-compatible endpoint, temperature (0.0), maximum output tokens (2,048), and automatic tool selection. The differences in timeout and retry settings concern provider-specific transport robustness, not task semantics or decoding parameters. Furthermore, all models achieve nearly identical TSR (99.55–100.00%), while their safety performance differs substantially. This distinction is supported by within-model intervention comparisons, not merely cross-model rankings. The common-eligibility analysis in Appendix D further shows that restricting CAFR to the same 89 eligible pairs preserves the complete model ranking, with a maximum change of only 0.76 percentage points. These results establish substantial differences in observable end-to-end safety adaptation under a standardized evaluation protocol.
W2 & Q4: Goal Preservation and Safe Feasibility.
Goal preservation is explicitly enforced by the benchmark construction rather than inferred from natural-language similarity. Each task is represented as τ=(u,g), where u is the user request and g is an executable goal predicate. Both contexts share the identical (u,g), while safety is independently evaluated through context-dependent violation predicates. As specified in Sections 3.2–3.6 and Appendix A.4, a pair is retained only when a safe goal-achieving realization exists in both contexts, verified through reference completion strategies and environment-state checks. For example, the request "let some fresh air in" preserves the functional goal of increasing cabin fresh air; opening a window and using an appropriate ventilation mechanism provide alternative realizations under different operating conditions. The reference strategies establish feasibility but are never exposed to evaluated agents. Thus, the evaluation distinguishes failure to adapt execution from changes in task semantics or inherent task infeasibility.
W2 & Q5: Safety Ground Truth and Simulation Validity.
Safety evaluation is defined through executable predicates over the current state, selected action, and resulting state transition, rather than subjective LLM judgments. As described in Appendix A.5, safety constraints have stable identifiers, applicable tasks, contextual triggering conditions, and rule provenance. Externally grounded requirements are distinguished from benchmark-specific operational assumptions. We will further clarify the individual rule conditions, thresholds, and supporting rationale to make these judgments directly auditable. CockpitWorld couples AAOS vehicle-function execution with CARLA driving-state evolution, propagating updated states across interaction steps. The four-step budget defines the evaluated interaction horizon; stateful execution and multi-step adaptation are supported within this budget. We will clarify this scope and improve the traffic depiction in Figure 1 and subplot readability in Figure 4.