acceptodds
Under review as a conference paper at ICLR 2027

E2M-Distill: Distilling End-to-End Driving Models into Motion-only Sim Agents for Large-Scale Autonomous Driving Simulation

Abstract

Widespread deployment of autonomous vehicles (AVs) may introduce unexpected system-level effects that require evaluation of interactions among multiple vehicles, so large-scale autonomous driving simulation is needed to understand their urban-mobility impact. Existing approaches face a trade-off: faithfully simulating real autonomous driving stacks from perception to planning is prohibitively expensive, while motion-only simulation bypasses the perceptual limitations of a target AV stack. We formulate this gap as *AV-Stack-to-Sim-Agent Conversion*: given a target AV stack, derive a motion-only sim agent (consuming only privileged simulator state, without rendering) whose closed-loop rollouts preserve the target stack's behavior. This task exhibits an *Inverted Learner-Expert Asymmetry* (ILEA): the teacher acts on partial sensor perception while the student sees privileged state, so conversion must account for the teacher's perceptual limitations. We propose **E2M-Distill** (E2E-to-Motion-only sim agent Distillation), a distillation framework that addresses ILEA. E2M-Distill places a Perception Contraction Module (PCM) ahead of the retained teacher planning head: PCM contracts the student's privileged simulator-state features toward the teacher's perception-conditioned representation. An interface-alignment loss anchors the contracted features to a teacher-side activation selected by the Perception-Planning Score (PPS). We introduce three closed-loop consistency metrics for this task and demonstrate across the evaluated target stacks that E2M-Distill improves behavioral fidelity while reducing the per-agent computational bottleneck for large-scale AV simulation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
autonomous driving, knowledge distillation, traffic simulation, privileged information, behavioral fidelity

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VnWLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes E2M-Distill, which converts an end-to-end autonomous driving system into a motion-only sim agent that relies only on privileged simulator state, reducing the computational cost of large-scale autonomous driving simulation. The method uses a Perception Contraction Module (PCM) to mimic the teacher's perception limitations, the Perception–Planning Score (PPS) to select the perception–planning interface, and an interface-alignment loss to preserve the teacher's behavioral characteristics. The authors evaluate the method on SparseDrive, VAD, and additional teacher settings, measuring open-loop trajectory error, closed-loop behavioral consistency, and inference latency.

Critical strengths
  1. The research problem is practically relevant. This paper studies how to convert a full autonomous driving system into a less computationally expensive sim agent while preserving the driving behavior of a given teacher as much as possible. This has clear practical value, especially for autonomous driving simulation involving large numbers of interacting vehicles.

  2. The paper provides a useful conceptualization of the teacher–student observation gap. The authors introduce ILEA to describe the asymmetry in autonomous driving model conversion, where the teacher relies on sensor observations with perception limitations, while the student has access to privileged simulator state. This perspective clarifies the specific challenges that traditional distillation approaches face in this task and provides a clear framework for understanding how differences in perception affect driving policy transfer.

Critical weaknesses
  1. Insufficient evidence for behavioral fidelity and incomplete efficiency comparisons. The main goal of this paper is to preserve the teacher's driving behavior, but the main closed-loop metrics measure agreement in route success or failure, correlation between driving scores, and overlap in infraction types. These metrics provide complementary information but cannot fully capture specific driving decisions, infraction severity, or when infractions occur. For example, a high correlation in driving scores does not mean that the teacher and student have similar absolute driving performance, and agreement in route outcomes does not necessarily mean that they exhibit similar driving behavior. Moreover, the VAD conversion shows clear degradation in open-loop trajectory prediction. E2M-Distill achieves trajectory errors of 17.9205 and 38.6938 at 0.5 s and 1.5 s, respectively, both substantially higher than those of the corresponding best baselines. Although the authors point out that the rankings of open-loop error and closed-loop consistency differ, this alone does not establish that the loss in open-loop performance is acceptable in practice. The current experiments therefore do not fully explain the relationship between open-loop trajectory degradation and improved closed-loop consistency, nor do they sufficiently establish that the current closed-loop metrics support the central claim of behavioral fidelity. In addition, the efficiency comparison is incomplete. Table 1 reports efficiency only for the original teachers and E2M-Distill, without providing system-level simulation latency under the same measurement protocol for baselines such as Output-only KD, FitNets KD, and Hand-crafted noise + KD. These results support the claim that removing the full perception pipeline reduces computational cost, but they do not establish an additional efficiency advantage over other student models. The student forward-pass latency tests in the appendix use different hardware and measurement scopes and do not cover the full simulation pipeline. Meanwhile, the teacher latency at the 100-vehicle scale is extrapolated rather than directly measured. These issues limit the comparability of the efficiency results across methods and weaken the evidence for the claimed efficiency gains in large-scale parallel simulation.

  2. Insufficient evidence for individual module contributions and a gap between the method's scope and its large-scale simulation claims. The ablation results show that removing IA or some distillation losses leads to performance degradation, but the individual contributions of the mechanisms within PPD are not sufficiently analyzed. PPD contains multiple degradation and gating mechanisms, while the current experiments mainly remove the entire module. This makes it difficult to distinguish the specific contributions and necessity of its individual components. Although the fixed-noise control provides some support for learned PPD, it does not fully reveal the individual contributions of its internal design choices. Meanwhile, the advantage of PPS over CKA-change is not consistent across teachers: on VAD, both interface-selection methods achieve the same SOA. The paper also does not sufficiently examine how interface selection is affected by calibration data, proxy tasks, or training randomness. As a result, the current experiments do not firmly establish the stable advantage of the proposed interface-selection strategy over simpler alternatives. Furthermore, E2M-Distill relies on access to the teacher's internal representations, an identifiable perception–planning interface, and an environment that provides synchronized privileged simulator state. These requirements limit its applicability to black-box driving systems and other architectures. Although the paper reports conversion results for different teacher models and inference latency at different numbers of parallel vehicles, the main behavioral evaluation still focuses on route-level teacher–student consistency. It does not sufficiently verify whether the converted agents preserve the teacher's characteristics in terms of interactions, traffic-flow distributions, and system-level effects in large mixed fleets. Therefore, the current results mainly support the feasibility of the method under the evaluated settings, while the independent value of its module designs and its applicability to broader architectures and large-scale interactive simulation remain uncertain.

Questions for authors
  1. Behavioral fidelity and open-loop degradation Given that E2M-Distill shows clear short-horizon open-loop trajectory error degradation on VAD while its closed-loop consistency metrics improve, how do the authors explain the difference between these results? Can the authors provide more direct behavioral evidence, such as actual planning-mode agreement or the timing of infractions, to demonstrate that the current closed-loop metrics adequately reflect the preservation of teacher behavior?

  2. Efficiency comparison Can the authors report full simulation latency and throughput for E2M-Distill, Output-only KD, FitNets KD, and Hand-crafted noise + KD using the same hardware and measurement protocol? In addition, since the teacher latency at the 100-vehicle scale is extrapolated, can the authors clarify the basis and uncertainty of the extrapolation to support the claims about large-scale simulation efficiency?

  3. Module contributions and PPS stability Can the authors further break down the contributions of the individual degradation and gating mechanisms within PPD and explain why PPS does not outperform CKA-change on VAD? Also, does the interface selected by PPS change with the calibration data, proxy tasks, or training random seeds?

  4. Applicability and system-level validation The current framework relies on access to the teacher's internal representations and an identifiable perception–planning interface. How do the authors define the scope of applicability of this method? Beyond route-level behavioral consistency and inference latency, is there evidence that the converted agents preserve the teacher's interaction behavior and system-level traffic effects in large mixed fleets?

4 comments

Thank you for your helpful comments. In response, we added direct behavioral comparisons and fleet-level traffic evaluations, measured full-simulation latency and throughput, analyzed PPD components and PPS sensitivity to calibration data, and clarified the method’s scope of applicability.

[Q1] Behavioral fidelity and open-loop degradation.

The two evaluations measure different aspects of fidelity. Open-loop error measures waypoint agreement, while closed-loop metrics summarize route outcomes, driving scores, and infraction types. Similar outcomes can arise from different trajectories and action timing, so improved closed-loop consistency alone does not establish accurate behavioral replay.

To provide more direct behavioral evidence, we evaluated SparseDriveV2 on the B2D validation set. We compared teacher and student predictions from the same scene states, using stop/go decisions and motion categories derived from their predicted trajectories. The motion categories include stopping, going straight, and turning left or right. These direct behavioral results are consistent with the gains in closed-loop metrics, supporting that these gains reflect better preservation of the teacher’s behavior.

MethodStop/go agreement (%)Teacher-stop recall (%)Motion-category agreement (%)
E2M-Distill86.1579.8875.67
Noise + KD81.4152.0568.40

[Q2] Efficiency comparison.

The student-forward timings in Appendix Table 7 show similar inference costs across student methods. To address the reviewer’s request for full-simulation measurements, we report results for the SparseDriveV2 teacher and two student models. All methods use the same H200 GPU, route, and measurement protocol. E2M-Distill achieves a 17.62× speedup in mean step latency over the teacher, while incurring additional overhead relative to the baseline.

MethodFull simulation (ms/step)Throughput (steps/s)
SparseDriveV2 teacher906.361.10
E2M-Distill51.4518.48
Noise + KD43.7622.14

The 100-vehicle teacher latencies in Table 1 are linearly extrapolated from measurements with up to 20 vehicles, while the student latencies are directly measured. Table 1 and the additional SparseDriveV2 experiments show over 10× speedups in single-vehicle simulation, and the students complete 100-vehicle closed-loop simulations within our resource budget. These results support the efficiency and scalability of the conversion without relying on the extrapolated teacher latencies.

[Q3] Module contributions and PPS stability

Due to time and resource constraints, we could not retrain the models for each ablation. We therefore disabled individual components at inference on the corrected checkpoints. The table shows the resulting increase in trajectory ADE relative to the teacher.

Component disabledSparseDriveV2 ΔADESparseDrive ΔADEVAD ΔADE
Feature gating+0.145+1.406+0.091
Anchor gating+0.720+0.045+0.025
All PPD+0.614+1.375+0.169

On VAD, PPS and CKA-change select the same transformer interface and use the same alignment target, which explains their identical downstream results. To address the reviewer’s concern, we repeated the comparison using two additional non-overlapping calibration sets. Across the original set and these two sets, PPS consistently selects fpn_convs.0 for SparseDrive and transformer for VAD. The other criteria change their selections in SparseDrive, while CKA-change is equally consistent on VAD.

[4] Applicability and system-level validation

E2M-Distill requires access to the teacher’s model architecture and intermediate representations. It therefore applies to E2E driving stacks that expose the necessary internal interfaces, while black-box systems without such access remain outside its current scope.

Following the reviewer’s suggestion, we added SparseDriveV2 fleet simulations and evaluated the system-level traffic metrics below. E2M-Distill is closer to the teacher than Noise + KD on four of the six metrics, while throughput and collision rates are identical across all three methods.

MetricSDv2 teacherE2M-DistillNoise + KD
Speed standard deviation (m/s)2.2692.8343.706
Peak speed loss ratio1.1581.3745.757
Throughput (veh/h)360360360
Collisions within the measurement section (events/veh-km)000
Hard deceleration (events/veh-km)11.90614.46315.307
Estimated energy consumption (kWh/100 veh-km)24.64833.25237.398
Reviewer_VnWLOpenReview ↗

Update Review

Thanks for detailed response. The additional experiments address some of my concerns. However, several issues remain insufficiently resolved.

1. Behavioral fidelity and open-loop degradation.

The new experiments provide more direct evidence of behavioral consistency, and the improvements over Noise + KD are encouraging. However, these experiments are limited to SparseDriveV2 and do not specifically analyze the reasons for the clear degradation in short-horizon open-loop trajectory errors on VAD. Although the authors explain that open-loop and closed-loop metrics measure different aspects of fidelity, the relationship between open-loop degradation and improved closed-loop consistency on VAD remains unclear.

2. Individual contributions of PPD components and PPS stability.

I appreciate the additional PPS calibration experiments and the explanation for why PPS and CKA-change achieve identical results on VAD. These results provide some evidence for the stability of PPS interface selection. However, the new PPD ablations only disable components of already-trained models at inference time. Such experiments mainly reflect the model's dependence on these components and may also be affected by distribution shifts caused by their removal. They cannot fully establish the individual contributions of these mechanisms during training. Although the original paper includes an ablation of the entire PPD module, the individual roles of its internal mechanisms remain unclear. In particular, noise injection and object dropout are already disabled at inference time, so their contributions have not been independently validated.

Overall, I acknowledge the authors' additional clarifications on efficiency evaluation, PPS stability, and the scope of applicability, and the new experiments alleviate some of my concerns. However, the open-loop degradation on VAD and the individual contributions of the PPD mechanisms still lack sufficient explanation and validation. Therefore, these additions do not fully resolve my concerns about the paper's main experimental conclusions.

[Q1] Behavioral fidelity and open-loop degradation.

We thank the reviewer for this point. Using the VAD weights from the paper, we further analyzed how trajectory errors translate into differences in control actions. The results indicate that larger waypoint deviations do not necessarily alter the current action; consequently, this can coexist with improvements in certain closed-loop consistency metrics.

Across the entire validation dataset, we compared the control outputs of the teacher model against those of the student model. Among the 7,095 frames where the 0.5-second trajectory error for E2M-Distill exceeded that of the Direct-GT baseline, there were still 2,298 frames (32.39%) in which the steering, throttle, and braking commands were identical to those of the teacher. We further compared the control errors of both student models relative to the teacher:

Comparison ResultFrame Count
E2M-Distill control is closer to the teacher521
Direct-GT control is closer to the teacher225
Control error is the same6,346
Each has its own pros and cons3

This indicates that, even though E2M-Distill exhibits a larger trajectory distance error, its control actions may still align with those of the teacher—and, in fact, be even closer to the teacher’s actions than those of the baseline.

We also examined 10 failure scenarios using the original closed-loop pipeline, comparing the policies along shared teacher-controlled prefixes at matched simulator states.

Metrics Relative to TeacherE2M-DistillDirect-GT
Steering MAE ↓0.06000.0638
Throttle MAE ↓0.29750.3175
Brake Consistency (%) ↑60.9257.90

In these closed-loop scenarios, the average control behavior of E2M-Distill is closer to that of the teacher model. On the 220 routes used in the paper, we also observed the higher closed-loop consistency in Table 1.

These results suggest that point-wise trajectory matching is not strictly equivalent to fidelity in execution behavior, although students exhibit some degradation in trajectory accuracy, E2M-Distill remains closer to the teacher model in terms of the control and closed-loop metrics evaluated.

[Q2] Individual contributions of PPD components.

Due to time and computational constraints, we could not retrain separate models for each PPD mechanism. We therefore added inference interventions to complement the original whole-PPD ablation. Although these interventions do not isolate training contributions, they provide evidence that the learned deterministic gates help maintain teacher planning fidelity in the deployed model. Together, these analyses with original PPD ablation support PPD’s contribution in the conversion framework.

Reviewer_ghGwOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This works aim to evlauate E2E driving stack w/ a surrogate of motion-only models. The key idea is to train a priviledged state input student that matches the target-policy in closed-loop. This is achieved by introducing perception contraction module, that models teacher's perception conditioned represenation. The experiments showed better close-loop metrics consistency compared to other approach.

Critical strengths

Very impactful idea if successful, most of the bottleneck of E2E driving is the evaluation, and it can be further expanded if similar ideas can adopt to traing hte E2E policy itself.

Critical weaknesses
  • The correlation of 0.6, and infraction shows that there are still much improvement in order to use E2EDistill in practice, what is the main bottleneck of this proposed method. It is important to seperate what failure mode (beahvior or perception) that E2EDistill can identify.
  • The privileged state of student should be a superset of teacher’s observation, in the current formulation, the student does not have information such as weather, or other important visual cues
Questions for authors
  • How does E2EDistill perform in long-tail scenarios? Such as near-collision, does the behavior resemble acutal E2E policy?
  • Same infraction does not mean same simulation, the E2EDistill can generate different collision to other cars than the E2E policy, can the authros provide more details on the breakdown of the Table 1? Does the E2EDistill have the same failure mode?
  • The paper will be stronger if authors provide more detailed trajecotry analysis of E2E closed-loop teacher beahvior, instead of the final metrics,
3 comments

We sincerely thank the reviewer for the constructive comments. Our current experiments mainly evaluate behavioral fidelity at the motion level. We agree that distinguishing behavioral and perception-related failure modes is important. In a small experiment, we changed image brightness and found that the teacher’s predictions changed while the student’s remained unchanged. Our current method does not capture such changes in visual appearance. We will revise the formulation and the corresponding descriptions in the paper.

We agree that final metrics alone do not clearly capture the details of teacher–student consistency. Examining behavior beyond the final metrics helps us identify the method’s limitations more clearly and guide further improvements. Following this suggestion, we expanded the SparseDriveV2 evaluation with more detailed comparisons of teacher and student behavior beyond the aggregate consistency metrics. Together, these results suggest that our method captures some of the teacher’s behavioral patterns and failure tendencies.

[Q1] E2EDistill performance in long-tail scenarios.

We first compared teacher–student agreement on collision and blocking outcomes using MCC and F1. Next, we added an analysis of SparseDriveV2 fleet simulations with 1, 3, 6, 9, and 12 controlled AVs. We compared fleets controlled by the teacher with those controlled by the student across 30 matched configurations. Together, these results suggest that E2M-Distill can capture the teacher’s long-tail behavior.

Failure typeE2M-Distill MCCNoise + KD MCCE2M-Distill F1 (%)Noise + KD F1 (%)
Collision0.20840.108333.9628.88
Blocking0.32580.319639.3938.36
Interaction metricTeacher–student Spearman correlation
Fraction with closing-TTC < 1.5 s0.673
Blocking rate0.885

[Q2] Details of failure mode.

We further analyzed the failure types underlying Table 1 and computed MCC and F1 for each type. Across the three teachers, E2M-Distill more closely matches the teacher’s failure patterns than the corresponding baseline in most comparisons.

Failure typeSD E2MSD Noise + KDVAD E2MVAD Direct-GTSDv2 E2MSDv2 Noise + KD
Vehicle collision0.165 / 34.620.093 / 29.630.073 / 29.350.043 / 28.110.225 / 34.720.165 / 30.95
Collision with static objects0.401 / 43.480.324 / 36.140.164 / 26.420.107 / 21.820.099 / 6.060.099 / 6.06
Blocking0.428 / 59.170.399 / 60.670.385 / 47.540.386 / 51.910.326 / 39.390.320 / 38.36
Outside route lanes0.368 / 53.730.269 / 46.970.029 / 22.22−0.031 / 16.280.046 / 8.330.082 / 10.17
Stop sign violation0.632 / 65.000.648 / 66.670.299 / 26.670.299 / 26.670.251 / 13.330.268 / 14.81
Route deviation————0.332 / 27.27−0.031 / 0.00

[Q3] Detailed closed-loop trajectory analysis.

Following the reviewer’s suggestion, we added a detailed closed-loop trajectory analysis using the same 30 paired fleet configurations. We retained the full 60 s of each trajectory and compared actual positions, speeds, and stop/go behavior with the teacher. Compared with Noise + KD, E2M-Distill reduces average position error by 8.28% and speed error by 23.15%, with lower speed error in 29 of the 30 configurations.

Metric over the full 60 sE2M-DistillNoise + KD
Teacher-relative position ADE (m) ↓40.2543.88
Speed MAE (m/s) ↓2.553.32
Stop/go agreement (%) ↑74.9168.98

Individual trajectories also show the behavioral differences more directly. For example, in a six-vehicle case in Town01, the teacher, E2M-Distill, and Noise + KD began a stop lasting at least 5 s at 14.6, 15.4, and 17.8 s, respectively. The teacher and E2M-Distill subsequently remained below 0.65 m/s, while Noise + KD accelerated again to 10.82 m/s.

Thank authors for the response, for Q1, it seems like there is a lot of difference betweeen closed-loop ADe compare to the E2E model (40 meters are quite large). I recommend the authors to provide more limitation and discussion on this topic, and how to close the gap (for example, is it due to vision input? Or some other treatment can potentially address the gap). Additionally, a discussion on the E2E distilll fidelity overtime (what horizon do we trust the results)

Thank you for these helpful suggestions. The 60 s closed-loop ADE shows that substantial trajectory differences remain despite the improvement over the baseline. Because small action differences may change subsequent states and interactions, leading to larger deviations over time, consistency between the student and teacher also decreases over time. For example, in the same experiments, their stop/go agreement rate decreases from 86.88% at 10 s to 83.44% at 20 s and 74.82% at 30 s. We will discuss this in the paper.

However, the student’s ability to reproduce the teacher’s behavior in specific scenarios does not appear to change systematically with driving time. We further examined this by selecting routes on which the student had a failure and dividing them into short-, medium-, and long-distance groups based on the distance traveled before the failure. For each route, we switched control of the vehicle from the student to the teacher before the original failure point. The resulting vehicle behavior showed no clear trend across the three groups. The three closed-loop consistency metrics in the paper are based on full-route completion outcomes and evaluate consistency from a different perspective than the analysis above, we will also further discuss the design of evaluation metrics for AV-Stack-to-Sim-Agent Conversion.

In addition, missing visual information is a clear limitation: the student cannot reproduce the teacher’s responses to appearance changes that are absent from its inputs, such as changes in weather. We will also discuss this limitation and analyse possible improvements.

Finally, thank you for your positive feedback on our work. We will expand our discussion and limitations in the camera-ready version if the paper is accepted. Your feedback provides valuable guidance for improving our work, and we will continue to pursue these improvements regardless of the final decision.

Reviewer_QzT5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper converts sensor-based driving models into cheaper simulation agents. It replaces visual perception with a state encoder and keeps the teacher’s planning head. It uses gating, training noise, and feature distillation to match the teacher’s behavior. On SparseDrive and VAD, it reports 14.9× and 26.1× speedups. The students also show better agreement with the teachers than the tested baselines.

Critical strengths
  1. The paper addresses a practical problem in driving simulation. It aims to reduce computation while preserving the behavior of a specific driving model. Avoiding image rendering offers substantial computational savings.

  2. Freezing the teacher’s planning head is a reasonable design choice. The student learns an interface from simulator states to the existing planning module. This provides a clear structure for converting different driving models.

  3. The evaluation covers multiple driving models and includes closed-loop tests. It measures agreement with the teacher across several metrics. The ablations also provide evidence for components.

Critical weaknesses

The student still differs substantially from the teacher. In Table 6, SparseDrive’s success rate increases from 47.27% to 70.45% after conversion. This is a large gap for a method intended to preserve behavior. Also it is not clear whether the student reproduces the same failures. More detailed comparisons will help.

It remains unclear whether PPD captures the teacher’s perception errors. Random noise and object dropout are disabled at inference. The training dropout is also not conditioned on the teacher’s actual missed detections. Controlled tests of occlusion and lighting would help distinguish this effect from regularization.

The gains over strong baselines are modest. Comparisons with existing perception simulation or perception-error models would help clarify the method’s contribution.

The large-scale experiments mainly demonstrate computational efficiency. They do not show whether a group of converted agents behaves like a group of original models. It would be good to test this in a small multi-AV setting.

Questions for authors

Please address questions in weaknesses.

5 comments

We sincerely thank the reviewer for the constructive comments. We added more detailed SparseDriveV2 comparisons of teacher–student behavior and failures, examined PPD under challenging visual conditions, and evaluated behavioral consistency in multi-AV settings.

[Q1] More detailed comparisons.

For cases where the teacher exhibits a collision or blocking failure, we measured how often the student exhibits the corresponding failure type. These results further support E2M-Distill’s ability to capture the teacher’s failure patterns.

Failure typeStudent occurrence rate when the teacher exhibits this failure
Collision81.82%
Blocking59.09%

We also examined whether the students retain the teacher’s driving decisions at matched scene states from the B2D validation set. We derived stop/go decisions and four motion categories—stopping, going straight, and turning left or right—from their predicted trajectories. E2M-Distill achieves higher agreement on all three measures. These comparisons identify stopping decisions as a specific aspect of teacher behavior that our method preserves more consistently.

MethodStop/go agreement (%)Teacher-stop recall (%)Motion-category agreement (%)
E2M-Distill86.1579.8875.67
Noise + KD81.4152.0568.40

[Q2] The effectiveness of PPD.

Random noise and object dropout are disabled at inference, while the learned deterministic gates remain active. The training dropout is not conditioned on the teacher’s actual missed detections. To examine the contribution of PPD during inference as suggested, we analyzed two cases under night and fog conditions. We compared E2M-Distill with Noise + KD and with PPD bypassed in the same trained checkpoint. The table reports trajectory ADE relative to the teacher.

CaseE2M-Distill ADE (m)Noise + KD ADE (m)PPD bypassed ADE (m)
Nighttime intersection with a vehicle1.9312.8805.182
Turn in fog with a pedestrian3.5123.6436.680

E2M-Distill matches the teacher more closely than Noise + KD in both cases, while bypassing PPD substantially increases the error. These results support the contribution of PPD module to planning fidelity in these cases.

[Q3] Gains over baselines and comparison with perception-error models.

The gains vary across metrics. The additional decision and fleet-trajectory comparisons in Q1 and Q4 help identify where E2M-Distill improves, particularly in stopping behavior and speed matching.

Existing perception simulation and perception-error models primarily emulate a target perception model’s outputs or error distribution, and also study their effects on downstream planning. These methods are closely related to ours, but their direct modeling targets differ. E2M-Distill focuses on converting a specified E2E driving model into a motion-only simulation agent by aligning simulator-state features with the teacher’s selected internal interface and retaining its planning-oriented head. Our goal is to preserve the teacher’s driving behavior after conversion. A direct comparison would therefore require adapting these methods to the same teacher interface. We will clarify this distinction and expand the discussion of these related methods in the paper.

[Q4] Behavioral consistency in small multi-AV settings.

We added SparseDriveV2 simulations with 1, 3, 6, 9, and 12 controlled AVs. Each fleet was controlled entirely by either the teacher or the student. Configurations with more blocking or more frequent low-TTC states for the teacher also tended to produce higher values for the student. This provides evidence that some interaction trends are retained across the fleet conditions.

Interaction metricTeacher–student Spearman correlation
Blocking rate0.885
Fraction with closing-TTC < 1.5 s0.673

We further compared how the vehicles moved over time, using 186 sets of corresponding vehicle trajectories from these configurations. Each comparison retained the full 60 s and measured position, speed, and stop/go differences relative to the teacher.

Metric over the full 60 sE2M-DistillNoise + KD
Teacher-relative position ADE (m) ↓40.2543.88
Speed MAE (m/s) ↓2.553.32
Actual stop/go agreement (%) ↑74.9168.98
Student stop rate while the teacher is moving (%) ↓29.8040.69

Compared with the baseline, E2M-Distill reduces average position error by 8.28% and speed error by 23.15%. It also follows the teacher’s stop/go behavior more closely. Together, these fleet statistics and trajectory comparisons provide evidence of behavioral preservation in the multi-AV settings.

Thanks for the additional experiments. The new failure analysis and multi-AV evaluations are helpful and address some of my concerns.

My main concern remains whether the student can reproduce the teacher's specific failures. The additional failure analysis is helpful, but the relatively low MCC and F1 scores might indicate that the student still struggles to reproduce the same failures. The large success-rate gap between the teacher and student also remains unexplained.

I still have some concerns about whether PPD captures the teacher's perception limitations.The night and fog examples show improved trajectory matching, but do not directly demonstrate that the student reproduces the teacher's perception errors.

Overall, the additional results strengthen the paper, but I still have concerns about its central claim of preserving the teacher's behavior.

Thank you for the constructive follow-up. To examine whether the student reproduces specific teacher failures, we added closed-loop replay experiments on teacher-failure scenarios from the original B2D test routes. We replayed the same teacher driving prefix and compared Teacher, E2M-Distill, and Noise + KD from verified matching handoff states. The following results summarize ten SparseDrive teacher-failure cases:

MetricE2M-DistillNoise + KD
Contact with the same object7/106/10
Same object, with matching motion states and pre-contact braking5/104/10
Reproduction of teacher blocking7/102/10
Stop/go agreement (%)62.0657.02

These results provide direct evidence that E2M-Distill can preserve aspects of the teacher’s specific failure behavior, including the collision object and nearby blocking, more faithfully than Noise + KD in these cases.

Regarding perception limitations, our current focus is the behavior of the complete end-to-end driving algorithm. We agree that improved trajectory matching in night and fog does not directly establish reproduction of the teacher’s perception errors. A detailed analysis linking object-level perception errors to their downstream planning consequences is a complementary perspective that we have not yet studied comprehensively. The present framework and these observations provide a basis and direction for that future investigation.

Overall, jointly improving computational efficiency and behavioral fidelity to the target AV stack is an important direction for AV-Stack-to-Sim-Agent Conversion. We believe E2M-Distill represents a useful step toward more accurate and efficient conversion. We hope this work encourages broader attention to this open problem and motivates more effective methods. We thank the reviewer for helping clarify these limitations and future directions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.