- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents TriDrive, a framework for forecasting driver behavior, vehicle dynamics, and road demands from synchronized cabin observations, vehicle signals, and road context. Its main design connects separately pretrained branches through directed residual connections, conditioned on the automation state. The evaluation covers pose forecasting on AIDE, event forecasting on BATON, and an additional set of unseen drivers. The authors also deploy the system on a vehicle and conduct a paired study with 14 participants. The deployed warning classifier uses current-state features, while the joint forecasting model runs alongside it.
Critical strengths
The forecasting evaluation is fairly thorough. The driver-disjoint BATON split and the frozen-model evaluation on 60 additional drivers and 42 non-overlapping vehicles are particularly useful. These tests provide reasonable evidence that the gains on selected forecasting targets extend beyond the original test split.
The deployment work adds practical value. The authors report measured on-vehicle performance, including a 5 Hz warning readout with 177 ms p95 latency while forecasting runs concurrently. They also report results that do not favor the method, including the lack of consistent warning gains from forecast features and the increased annoyance in the user study. This makes the contribution and its limits easier to assess.
Critical weaknesses
My main reservation is the evidence for the joint architecture itself. The AIDE result comes from a separate pose forecaster, and the deployed warning classifier does not use the joint model’s forecasts. Adding forecast features does not consistently improve warning AUROC (Appendix D.2). The BATON results do support useful forecasting gains, but all-to-all fusion is competitive and sometimes better in Table 3. The paper would be stronger if it stated more clearly which benefits require the proposed directed connections, and which come from the input representation or training recipe. At present, I find the empirical contribution stronger than the architectural novelty.
The warning results are encouraging, but the practical advantage remains qualified. The offline openpilot-based baseline learns to reproduce logged alerts rather than replaying the actual policy, which complicates the comparison. The advantage also reverses under assistance-engaged driving: AUROC is 0.664 versus 0.746 for the baseline. In the on-road study, appropriateness and timeliness improve, but annoyance increases and overall preference remains inconclusive. Per-participant driving exposure and warning rates would help interpret these results, especially given the different aggregate recording durations reported in Appendix H
Questions for authors
Do you have an existing comparison between directed connections and an equally sized fusion baseline using the same inputs and training schedule? This would help clarify the benefit of the connection design beyond its smaller parameter count relative to the reported all-to-all model.
How closely does the offline openpilot-based surrogate reproduce actual logged alerts on held-out drivers? If this has been measured, please report it so that the warning comparison can be interpreted more directly.
From the existing study logs, could you provide the paired driving durations and warning rates for each participant? Please also clarify how the 5.9 versus 2.5 hours of device recordings relate to the drives included in the paired analysis, and whether alert sound and display settings were matched.
2 comments
Dear Reviewer bdt1,
Thank you again for your constructive review. Our response is posted above, together with a revised PDF.
In brief, we added parameter-matched fusion and training-objective controls to clarify the source of the forecasting gains (Table 10). We also clarified the offline surrogate’s prediction targets (Table 19) and distinguished this comparison from the actual openpilot system used in the on-road study. Finally, we explained how the aggregate recording durations relate to the participant study, documented the matched alert presentation and threshold-calibration procedure, and provided links to the deployment materials and session logs.
Could you let us know whether these revisions address your concerns, or which points remain open? We will respond to any further questions before the discussion closes on November 18 (AoE). If the concerns are resolved, we would be grateful if you could reflect this in your assessment.
Best regards,
The authors
Thank you for the constructive assessment. We have added matched fusion controls, measured surrogate fidelity, and clarified the on-road study.
1. Comparing directed connections with equally sized fusion.
We added a concatenation control combining driver and road states with the vehicle latent, using 575,616 versus 576,768 connection parameters for directed fusion, a 0.2% difference. Inputs, pretrained branches, training schedule, model selection, and minibatch sequence are matched across three seeds.(Appendix B.6, Table 10).
The control improves PR-AUC over independent branches by 0.037/0.028/0.042 for manual steering/gas/brake and 0.079 for engaged steering, versus 0.030/0.020/0.037 and 0.084 for directed fusion. Concatenation is higher by 0.007 ± 0.002 on manual steering; other designated benchmark differences do not meet our criterion. On 60 additional drivers, its steering gains are 0.049/0.072 over independent branches, with neither steering difference from directed fusion meeting the criterion.
Simpler fusion can therefore integrate this information effectively. We retain directed connections because driver actions and road demands inform vehicle motion, and explicit paths preserve separate driver/road predictions with 29.8% fewer connection parameters than all-to-all. Fine-tuning with vehicle losses alone changes the designated vehicle scores by at most 0.001. Cross-domain integration drives the vehicle gains, while joint supervision maintains all three tasks without a detectable vehicle-score penalty. We therefore favor directed connections for deployment, as they offer competitive vehicle forecasting performance with fewer connection parameters than all-to-all coupling.
Sections 4.2–4.3 now attribute AIDE gains to anchored kinematics and pretraining, vehicle-onset gains to cross-domain connections, and TTC-drop gains to structured road margins. Figure 4 and Appendix I separately evaluate vehicle information through frozen-feature readouts.
2. Fair comparison with openpilot.
Table 19 evaluates the surrogate’s attention-state predictions, rather than exact agreement with logged alerts. Our on-road study directly compares against openpilot’s actual driver-monitoring system, independently of this surrogate.
For a fair on-road comparison, we selected both systems’ operating thresholds on offline test recordings under the same nominal warning-budget target, then fixed them before the participant drives. For example, with a target of 20 warnings per hour, we adjust each system’s threshold on the same offline recordings to approach that rate, then keep both thresholds fixed during the participant drives.
Both systems therefore followed the same calibration rule. Under this protocol, TriDrive received higher paired ratings for appropriateness (+1.79) and timeliness (+2.67) (Section 4.4, Figure 5).
3. Study recordings and matched alert settings.
The 5.9 versus 2.5 hours describe the complete recording archive, including sessions before the questionnaire study and instrumented drives. They are not the paired driving durations used in the questionnaire comparison. Each participant drove the same route in their own vehicle under both conditions, with condition order alternated. The device logs also provide session-level driving durations, counting windows, and counts of presented level-2 prompts.
Both conditions used identical comma four devices and the same openpilot build. TriDrive reused openpilot’s own level-2 attention alert, so the text, display size, and alert sound were matched; level-3 escalation remained unchanged.
All the data, log and how we implemented TriDrive on comma is released on the Hugging Face repo: https://huggingface.co/AsianPlayer/TriDrive. You can also check in the supplimentary material. We will also make the full-modal data from the real-world vehicle testing (including driving video, driver video, audio, and CAN logs) publicly available after publication.
Thank you.