- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes E2M-Distill, which converts an end-to-end autonomous driving system into a motion-only sim agent that relies only on privileged simulator state, reducing the computational cost of large-scale autonomous driving simulation. The method uses a Perception Contraction Module (PCM) to mimic the teacher's perception limitations, the Perception–Planning Score (PPS) to select the perception–planning interface, and an interface-alignment loss to preserve the teacher's behavioral characteristics. The authors evaluate the method on SparseDrive, VAD, and additional teacher settings, measuring open-loop trajectory error, closed-loop behavioral consistency, and inference latency.
Critical strengths
-
The research problem is practically relevant. This paper studies how to convert a full autonomous driving system into a less computationally expensive sim agent while preserving the driving behavior of a given teacher as much as possible. This has clear practical value, especially for autonomous driving simulation involving large numbers of interacting vehicles.
-
The paper provides a useful conceptualization of the teacher–student observation gap. The authors introduce ILEA to describe the asymmetry in autonomous driving model conversion, where the teacher relies on sensor observations with perception limitations, while the student has access to privileged simulator state. This perspective clarifies the specific challenges that traditional distillation approaches face in this task and provides a clear framework for understanding how differences in perception affect driving policy transfer.
Critical weaknesses
-
Insufficient evidence for behavioral fidelity and incomplete efficiency comparisons. The main goal of this paper is to preserve the teacher's driving behavior, but the main closed-loop metrics measure agreement in route success or failure, correlation between driving scores, and overlap in infraction types. These metrics provide complementary information but cannot fully capture specific driving decisions, infraction severity, or when infractions occur. For example, a high correlation in driving scores does not mean that the teacher and student have similar absolute driving performance, and agreement in route outcomes does not necessarily mean that they exhibit similar driving behavior. Moreover, the VAD conversion shows clear degradation in open-loop trajectory prediction. E2M-Distill achieves trajectory errors of 17.9205 and 38.6938 at 0.5 s and 1.5 s, respectively, both substantially higher than those of the corresponding best baselines. Although the authors point out that the rankings of open-loop error and closed-loop consistency differ, this alone does not establish that the loss in open-loop performance is acceptable in practice. The current experiments therefore do not fully explain the relationship between open-loop trajectory degradation and improved closed-loop consistency, nor do they sufficiently establish that the current closed-loop metrics support the central claim of behavioral fidelity. In addition, the efficiency comparison is incomplete. Table 1 reports efficiency only for the original teachers and E2M-Distill, without providing system-level simulation latency under the same measurement protocol for baselines such as Output-only KD, FitNets KD, and Hand-crafted noise + KD. These results support the claim that removing the full perception pipeline reduces computational cost, but they do not establish an additional efficiency advantage over other student models. The student forward-pass latency tests in the appendix use different hardware and measurement scopes and do not cover the full simulation pipeline. Meanwhile, the teacher latency at the 100-vehicle scale is extrapolated rather than directly measured. These issues limit the comparability of the efficiency results across methods and weaken the evidence for the claimed efficiency gains in large-scale parallel simulation.
-
Insufficient evidence for individual module contributions and a gap between the method's scope and its large-scale simulation claims. The ablation results show that removing IA or some distillation losses leads to performance degradation, but the individual contributions of the mechanisms within PPD are not sufficiently analyzed. PPD contains multiple degradation and gating mechanisms, while the current experiments mainly remove the entire module. This makes it difficult to distinguish the specific contributions and necessity of its individual components. Although the fixed-noise control provides some support for learned PPD, it does not fully reveal the individual contributions of its internal design choices. Meanwhile, the advantage of PPS over CKA-change is not consistent across teachers: on VAD, both interface-selection methods achieve the same SOA. The paper also does not sufficiently examine how interface selection is affected by calibration data, proxy tasks, or training randomness. As a result, the current experiments do not firmly establish the stable advantage of the proposed interface-selection strategy over simpler alternatives. Furthermore, E2M-Distill relies on access to the teacher's internal representations, an identifiable perception–planning interface, and an environment that provides synchronized privileged simulator state. These requirements limit its applicability to black-box driving systems and other architectures. Although the paper reports conversion results for different teacher models and inference latency at different numbers of parallel vehicles, the main behavioral evaluation still focuses on route-level teacher–student consistency. It does not sufficiently verify whether the converted agents preserve the teacher's characteristics in terms of interactions, traffic-flow distributions, and system-level effects in large mixed fleets. Therefore, the current results mainly support the feasibility of the method under the evaluated settings, while the independent value of its module designs and its applicability to broader architectures and large-scale interactive simulation remain uncertain.
Questions for authors
-
Behavioral fidelity and open-loop degradation Given that E2M-Distill shows clear short-horizon open-loop trajectory error degradation on VAD while its closed-loop consistency metrics improve, how do the authors explain the difference between these results? Can the authors provide more direct behavioral evidence, such as actual planning-mode agreement or the timing of infractions, to demonstrate that the current closed-loop metrics adequately reflect the preservation of teacher behavior?
-
Efficiency comparison Can the authors report full simulation latency and throughput for E2M-Distill, Output-only KD, FitNets KD, and Hand-crafted noise + KD using the same hardware and measurement protocol? In addition, since the teacher latency at the 100-vehicle scale is extrapolated, can the authors clarify the basis and uncertainty of the extrapolation to support the claims about large-scale simulation efficiency?
-
Module contributions and PPS stability Can the authors further break down the contributions of the individual degradation and gating mechanisms within PPD and explain why PPS does not outperform CKA-change on VAD? Also, does the interface selected by PPS change with the calibration data, proxy tasks, or training random seeds?
-
Applicability and system-level validation The current framework relies on access to the teacher's internal representations and an identifiable perception–planning interface. How do the authors define the scope of applicability of this method? Beyond route-level behavioral consistency and inference latency, is there evidence that the converted agents preserve the teacher's interaction behavior and system-level traffic effects in large mixed fleets?
4 comments
[Q3] Module contributions and PPS stability
Due to time and resource constraints, we could not retrain the models for each ablation. We therefore disabled individual components at inference on the corrected checkpoints. The table shows the resulting increase in trajectory ADE relative to the teacher.
| Component disabled | SparseDriveV2 ΔADE | SparseDrive ΔADE | VAD ΔADE |
|---|---|---|---|
| Feature gating | +0.145 | +1.406 | +0.091 |
| Anchor gating | +0.720 | +0.045 | +0.025 |
| All PPD | +0.614 | +1.375 | +0.169 |
On VAD, PPS and CKA-change select the same transformer interface and use the same alignment target, which explains their identical downstream results. To address the reviewer’s concern, we repeated the comparison using two additional non-overlapping calibration sets. Across the original set and these two sets, PPS consistently selects fpn_convs.0 for SparseDrive and transformer for VAD. The other criteria change their selections in SparseDrive, while CKA-change is equally consistent on VAD.
[4] Applicability and system-level validation
E2M-Distill requires access to the teacher’s model architecture and intermediate representations. It therefore applies to E2E driving stacks that expose the necessary internal interfaces, while black-box systems without such access remain outside its current scope.
Following the reviewer’s suggestion, we added SparseDriveV2 fleet simulations and evaluated the system-level traffic metrics below. E2M-Distill is closer to the teacher than Noise + KD on four of the six metrics, while throughput and collision rates are identical across all three methods.
| Metric | SDv2 teacher | E2M-Distill | Noise + KD |
|---|---|---|---|
| Speed standard deviation (m/s) | 2.269 | 2.834 | 3.706 |
| Peak speed loss ratio | 1.158 | 1.374 | 5.757 |
| Throughput (veh/h) | 360 | 360 | 360 |
| Collisions within the measurement section (events/veh-km) | 0 | 0 | 0 |
| Hard deceleration (events/veh-km) | 11.906 | 14.463 | 15.307 |
| Estimated energy consumption (kWh/100 veh-km) | 24.648 | 33.252 | 37.398 |
Update Review
Thanks for detailed response. The additional experiments address some of my concerns. However, several issues remain insufficiently resolved.
1. Behavioral fidelity and open-loop degradation.
The new experiments provide more direct evidence of behavioral consistency, and the improvements over Noise + KD are encouraging. However, these experiments are limited to SparseDriveV2 and do not specifically analyze the reasons for the clear degradation in short-horizon open-loop trajectory errors on VAD. Although the authors explain that open-loop and closed-loop metrics measure different aspects of fidelity, the relationship between open-loop degradation and improved closed-loop consistency on VAD remains unclear.
2. Individual contributions of PPD components and PPS stability.
I appreciate the additional PPS calibration experiments and the explanation for why PPS and CKA-change achieve identical results on VAD. These results provide some evidence for the stability of PPS interface selection. However, the new PPD ablations only disable components of already-trained models at inference time. Such experiments mainly reflect the model's dependence on these components and may also be affected by distribution shifts caused by their removal. They cannot fully establish the individual contributions of these mechanisms during training. Although the original paper includes an ablation of the entire PPD module, the individual roles of its internal mechanisms remain unclear. In particular, noise injection and object dropout are already disabled at inference time, so their contributions have not been independently validated.
Overall, I acknowledge the authors' additional clarifications on efficiency evaluation, PPS stability, and the scope of applicability, and the new experiments alleviate some of my concerns. However, the open-loop degradation on VAD and the individual contributions of the PPD mechanisms still lack sufficient explanation and validation. Therefore, these additions do not fully resolve my concerns about the paper's main experimental conclusions.
[Q1] Behavioral fidelity and open-loop degradation.
We thank the reviewer for this point. Using the VAD weights from the paper, we further analyzed how trajectory errors translate into differences in control actions. The results indicate that larger waypoint deviations do not necessarily alter the current action; consequently, this can coexist with improvements in certain closed-loop consistency metrics.
Across the entire validation dataset, we compared the control outputs of the teacher model against those of the student model. Among the 7,095 frames where the 0.5-second trajectory error for E2M-Distill exceeded that of the Direct-GT baseline, there were still 2,298 frames (32.39%) in which the steering, throttle, and braking commands were identical to those of the teacher. We further compared the control errors of both student models relative to the teacher:
| Comparison Result | Frame Count |
|---|---|
| E2M-Distill control is closer to the teacher | 521 |
| Direct-GT control is closer to the teacher | 225 |
| Control error is the same | 6,346 |
| Each has its own pros and cons | 3 |
This indicates that, even though E2M-Distill exhibits a larger trajectory distance error, its control actions may still align with those of the teacher—and, in fact, be even closer to the teacher’s actions than those of the baseline.
We also examined 10 failure scenarios using the original closed-loop pipeline, comparing the policies along shared teacher-controlled prefixes at matched simulator states.
| Metrics Relative to Teacher | E2M-Distill | Direct-GT |
|---|---|---|
| Steering MAE ↓ | 0.0600 | 0.0638 |
| Throttle MAE ↓ | 0.2975 | 0.3175 |
| Brake Consistency (%) ↑ | 60.92 | 57.90 |
In these closed-loop scenarios, the average control behavior of E2M-Distill is closer to that of the teacher model. On the 220 routes used in the paper, we also observed the higher closed-loop consistency in Table 1.
These results suggest that point-wise trajectory matching is not strictly equivalent to fidelity in execution behavior, although students exhibit some degradation in trajectory accuracy, E2M-Distill remains closer to the teacher model in terms of the control and closed-loop metrics evaluated.
[Q2] Individual contributions of PPD components.
Due to time and computational constraints, we could not retrain separate models for each PPD mechanism. We therefore added inference interventions to complement the original whole-PPD ablation. Although these interventions do not isolate training contributions, they provide evidence that the learned deterministic gates help maintain teacher planning fidelity in the deployed model. Together, these analyses with original PPD ablation support PPD’s contribution in the conversion framework.
Thank you for your helpful comments. In response, we added direct behavioral comparisons and fleet-level traffic evaluations, measured full-simulation latency and throughput, analyzed PPD components and PPS sensitivity to calibration data, and clarified the method’s scope of applicability.
[Q1] Behavioral fidelity and open-loop degradation.
The two evaluations measure different aspects of fidelity. Open-loop error measures waypoint agreement, while closed-loop metrics summarize route outcomes, driving scores, and infraction types. Similar outcomes can arise from different trajectories and action timing, so improved closed-loop consistency alone does not establish accurate behavioral replay.
To provide more direct behavioral evidence, we evaluated SparseDriveV2 on the B2D validation set. We compared teacher and student predictions from the same scene states, using stop/go decisions and motion categories derived from their predicted trajectories. The motion categories include stopping, going straight, and turning left or right. These direct behavioral results are consistent with the gains in closed-loop metrics, supporting that these gains reflect better preservation of the teacher’s behavior.
[Q2] Efficiency comparison.
The student-forward timings in Appendix Table 7 show similar inference costs across student methods. To address the reviewer’s request for full-simulation measurements, we report results for the SparseDriveV2 teacher and two student models. All methods use the same H200 GPU, route, and measurement protocol. E2M-Distill achieves a 17.62× speedup in mean step latency over the teacher, while incurring additional overhead relative to the baseline.
The 100-vehicle teacher latencies in Table 1 are linearly extrapolated from measurements with up to 20 vehicles, while the student latencies are directly measured. Table 1 and the additional SparseDriveV2 experiments show over 10× speedups in single-vehicle simulation, and the students complete 100-vehicle closed-loop simulations within our resource budget. These results support the efficiency and scalability of the conversion without relying on the extrapolated teacher latencies.