acceptodds
Under review as a conference paper at ICLR 2027

SEAM: Shared-Expert Adaptation Masked by Closed-Loop Scores in Reward Fine-Tuned Diffusion Planners

Abstract

Reward fine-tuning is an appealing way to optimize safety and comfort in diffusion-based driving planners, but a higher training reward need not mean better closed-loop driving. We fine-tune the Diffusion Planner by backpropagating through the 10-step DPM-Solver++ that it runs at deployment, on two independent stacks, Autoware and nuPlan. The optimized training reward rises by a factor of 2.5, yet on the nuPlan val14 closed-loop score no fine-tuned configuration improves on the behavior-cloning checkpoint beyond seed variation. The planner is nonetheless learning. Using Shared-Expert LoRA (SE-LoRA), which adds one low-rank adapter shared by all driving domains and one adapter per domain, we show that the adaptation is structured. Domains with little training data are carried by the shared adapter, while domains with more data rely increasingly on their own. Removing the shared adapter collapses the smallest domain even though more parameters are trained, and removing the per-domain adapters costs the largest domain 15 percent. The closed-loop score misses this because the per-scenario changes are small, point both ways, and barely change how many scenarios pass or fail the 0/1 gates that the score is built from. For reward fine-tuning, this means a rising training reward is not evidence of better driving, and a flat closed-loop score is not evidence that nothing was learned.

Primary area:
applications to robotics, autonomy, planning
Keywords:
diffusion models, reward fine-tuning, autonomous driving, trajectory planning, low-rank adaptation, multi-domain learning, interpretability, closed-loop evaluation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Eic1OpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates reward fine-tuning for diffusion-based autonomous driving planners. It combines shared and domain-specific LoRA adapters, differentiable DPM-Solver++ sampling, and reward diagnostics to analyze why higher training rewards do not translate into improved closed-loop driving performance.

Critical strengths

The paper investigates a relevant problem in diffusion planner fine-tuning and provides several diagnostic experiments.

Critical weaknesses
  1. Limited novelty and comparison.
    The proposed SE-LoRA is essentially a combination of shared and domain-specific LoRA adapters, while differentiating through an unrolled diffusion solver is not fundamentally new. More importantly, the lack of comparison with any of the other methods on NuPlan makes the performance hard to appreciate.
  2. Ambiguous experiment.
    The paper demonstrates that reward fine-tuning changes the planner's behavior (Lines 408–413), but provides little evidence that these changes are beneficial. As shown in Table 2, no fine-tuned configuration (row 1) achieves a meaningful closed-loop improvement, and some introduce additional catastrophic failures. While the authors analyze why the aggregate score remains unchanged, it is unclear whether the observed adaptations represent useful driving capabilities rather than simply changes in model behavior.
  3. Insufficient documentation of real-vehicle experiments.
    The claimed real-vehicle deployment is also questionable due to the lack of experimental photographs and a sufficiently detailed testing setup. It remains unclear how the experiments were actually conducted and whether any real-world closed-loop driving evaluation was performed, making the claimed deployment difficult to assess or verify.
Questions for authors

See weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewer Eic1: Measurement Positioning, nuPlan Baselines, Route-Progress Gains, and Real-Vehicle Setup

Thanks for your review.

W1. Novelty, positioning, and comparison with published nuPlan planners

  • Measurement contribution rather than a new update rule (Sec. 1, 2): We agree that shared/per-domain LoRA and differentiating through an unrolled solver are known techniques, and claim neither as an algorithmic novelty. We use them as diagnostic instruments to study why held-out training rewards rise by 30% to 60% while val14 closed-loop scores do not (Table 2). Unlike reward fine-tuning papers reporting benchmark average gains (DPPO, DiffusionDriveV2, PlannerRFT, Plan-R1), we analyze the flat-average case via scenario-by-scenario comparisons against repeated BC simulations and gate vs. component splits (Sec. 4.4, App. C.11). Unlike learned-router MoE-LoRA (MoLA, MixLoRA, MoLoRA), SE-LoRA routes by map so either half can be removed to measure what it carried (Sec. 4.3, Figure 2).
  • Comparison on nuPlan: The three simulations of the base planner in our pipeline are within 0.1 points of its published score on val14 and within 1 point on the two Test14 lists of the official test split (our runs are in Sec. 4.2, Table 6, and App. C.12; the last three rows below are quoted from Table 1 of the base planner's paper and were not run by us; 0 to 100 non-reactive scale):
Plannerval14Test14-randomTest14-hard
Base planner, as published89.8789.1975.99
Base planner in our pipeline (3 runs)89.83, 89.77, 89.7989.26, 88.57, 88.6775.08, 75.25, 75.68
Fine-tuned, exact operators (3 seeds)89.20, 89.27, 89.3088.62, 88.82, 88.4575.89, 75.55, 75.87
PDM-Closed, rule-based (as published)92.8490.0565.08
PLUTO w/o refinement (as published)88.8989.9070.03
PlanTF (as published)84.2785.6269.70

Other Table 2 fine-tuned rows score 88.4 to 89.7 on val14.

W2. Are the changes beneficial, or only changes in behavior?

With exact score operators, the fine-tuned planners of Table 2 complete more of their route in every seed on val14 and on disjoint Test14-random, offset by added gate failures (Sec. 4.4, Table 6, App. C.11):

Three training seeds vs. 3 BC runsval14 (1,118 scenarios)Test14-random (261, disjoint logs)
Route-progress change (0 to 1 scale)+0.0053, +0.0051, +0.0056+0.0039, +0.0038, +0.0040 (all 95% CIs > 0)
Newly perfect (≥0.99) / lost34, 37, 40 / 6, 6, 75, 6, 5 / 3, 3, 4
Newly zero / left zero10, 11, 9 / 1, 2, 12, 2, 3 / 2, 1, 2
Largest newly perfect / newly zero counts between BC runs2 / 11 / 1

Nearly all newly perfect val14 scenarios (32 shared across all 3 seeds) gain via route progress alone, and Test14-random meets pre-fixed criteria (Test14-hard progress also rises in all seeds, CI > 0 in one; a 169-scene validation hold-out does not rise). Because a smaller group fails collision or drivable-area gates (min TTC shortens by ~0.03 s), val14 ends ~0.5 points lower (0 to 100 scale) and Test14 is indistinguishable from BC. Thus we claim a route-progress gain with no net score gain, and removed "safety improvement" wording.

W3. Was a closed-loop real-vehicle drive performed, and how was it set up?

  • Closed-loop base drive (App. D): Yes. The base planner drove our drive-by-wire research vehicle in closed loop inside Autoware over a 704 m stop-and-go route (mean speed 1.25 m/s, safety driver aboard), using one 128-beam rotating LiDAR, a dual-antenna INS, and an onboard laptop (GPU perception, CPU diffusion planning; App. D.1 lists all 4 stack changes and the active safety layer).
  • Offline SE-LoRA comparison (Table 24): An earlier SE-LoRA checkpoint (suburban map adapter) did not control the vehicle. Its first exported network produced implausible speeds during the field test and was re-exported afterwards, which is why the comparison is an offline replay. Replaying both models offline across 11,577 paired sensor frames from two recorded drives (open loop, no tracked objects) gives a maximum difference of 2.1% across four trajectory statistics. Main-text claims rest on Section 4, while Appendix D verifies execution on real sensor input. Photographs of the vehicle and test site are omitted from the anonymous version and will be added to the final version.

I appreciate the authors' responses; I still have the following questions:

  1. After carefully reading, I'm still confused about the evaluation setting and metrics used in the paper. Could the authors provide the leaderboard results under NuPlan in a standard benchmarking setting and compare them with other methods? The new table is appreciated, but I didn't see it in the revised paper, and it is missing some of the comparisons, e.g., as you mentioned: DPPO, DiffusionDriveV2, PlannerRFT, Plan-R1.
  2. If it is only deployed on Autoware virtually, it should not be called a real-vehicle experiment; moreover, I would like to see more visualization results of common failure cases that previous methods may have, as well as a short video under Autoware.
  3. The writing of the paper is barely understandable; although it contains a lot of experiments, it is still really hard to read and seems like a "report" rather than a paper.

I raised my score given the author's current responses, while some of my concerns are still unresolved, as I mentioned.

AuthorsOpenReview ↗

Response to Reviewer Eic1's follow-up: Standard nuPlan Table, Real-Vehicle Setup, Failure Plots, and Writing

Thank you for raising your score and for the follow-up questions. The additions below are in the updated PDF, marked "Eic1 follow-up" in blue.

1. Standard nuPlan setting and comparison table

Our nuPlan closed-loop experiments use the official nuPlan non-reactive benchmark on the standard val14, Test14-random, and Test14-hard splits (plus one 169-scenario hold-out, App. C.13). Appendix G, Table 26 now provides the comparison on the standard 0–100 leaderboard scale, with published baselines quoted from the literature (sources are listed in Table 26):

Plannerval14Test14-randomTest14-hard
Base planner, as published89.8789.1975.99
Base planner in our pipeline, 3 runs89.77 to 89.8388.57 to 89.2675.08 to 75.68
Fine-tuned, exact score operators, 3 seeds89.20 to 89.3088.45 to 88.8275.55 to 75.89
PlannerRFT, as published89.9690.7677.16
Plan-R1, as published88.9891.2377.45
Plan-R1 with rule-based post-processing94.7294.6478.46
PDM-Closed, rule-based92.8490.0565.08
PLUTO w/o refinement88.8989.9070.03
PlanTF84.2785.6269.70

DPPO is evaluated on robotics tasks and DiffusionDriveV2 on NAVSIM, so neither reports nuPlan scores; our own adapted DPPO run scores 89.60 on val14 with a flat reward. Rather than proposing a leaderboard planner, our paper studies rows 2 and 3: why a 30%–35% gain in held-out reward (up to 60% with our other rewards) leaves the closed-loop score flat or slightly lower.

2. Autoware and the real-vehicle test

Autoware itself is an open-source autonomous driving software stack, not a simulator: while our multi-map ablations use it without a vehicle, in Appendix D it connected directly to our physical drive-by-wire vehicle, processing live LiDAR and INS data to send real control commands. The base planner drove the physical car in closed loop over a 704 m route with a safety driver aboard. However, the fine-tuned planner never drove the car; its comparison with the base planner is an offline replay on sensor logs recorded on the vehicle. We agree that this offline replay should not be called a real-vehicle experiment, and we updated Appendix D's title and Appendix C.9 accordingly.

3. Failure case visualizations and video

We added bird's-eye-view plots of four zero-score failure cases (four of the six we simulated again) in Appendix H, Figure 5, covering traversing pickup dropoff, changing lane, and stopping with lead. Comparing both planners at the exact failure step shows the fine-tuned planner slightly further ahead along the route, where it clips a stopped vehicle, is touched by a pedestrian while still moving, or puts a corner outside the drivable area. We cannot record a video in the remaining hours and will add a supplementary Autoware video in the final version.

4. Writing and readability

Thank you for the candid feedback. Because this is a measurement study with many ablations, the initial draft read too much like a technical report. In the revised PDF, we reorganized the main text around one central question, rewrote the results subsection headings to state conclusions directly, and moved detailed protocols to the appendix. We will further trim secondary numbers from the body text in the final version.

Reviewer_4gtKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SEAM studies reward fine-tuning of diffusion driving planners using shared and domain-specific LoRA adapters and gradients through the deployed sampling solver. Training reward increases substantially, while the reported nuPlan closed-loop aggregate score remains within seed variation. Adapter ablations and gate analyses are used to investigate changes that the aggregate score may obscure. Experiments cover Autoware and nuPlan.

Critical strengths
  1. Reporting the negative closed-loop result with seed variation and paired equivalence testing is useful. The recommendation to inspect per-scenario scores and gate flips can improve the interpretation of reward fine-tuning experiments.

  2. Differentiating through the deployed solver removes a concrete sampler-mismatch confound. Studying two stacks and transparently correcting an earlier analysis also strengthen the methodological presentation.

Critical weaknesses
  1. The evidence needs to distinguish a recipe-specific negative result from a broader measurement finding. The narrow adapter placement, optimization budget and reward design may explain the flat closed-loop score. Prior work reporting RL gains does not invalidate this negative result, but it makes an in-pipeline comparison or more precise statement of scope important. The familiar shared/private adapter behavior alone adds limited insight into why driving improvements would be masked.

  2. The claim of masked beneficial learning lacks a clear held-out demonstration. Increased training reward and changed trajectories establish adaptation, but not improved driving on unseen scenarios. Continuous held-out safety/comfort metrics and a per-component score decomposition are needed to distinguish useful improvements from reward fitting or trade-offs. The fine-tuning/evaluation split and inconsistent scenario counts also require clarification before the statistical result can be interpreted confidently. I currently lean toward weak rejection.

Questions for authors
  1. What is the log- and scenario-level overlap between fine-tuning data and val14? Can the key analysis be reported on a demonstrably disjoint hold-out set? Please reconcile n=1133 with the stated val14 scenario count and identify which comparisons use multiple seeds.

  2. On held-out scenes, which continuous safety, collision-margin, RSS or comfort measures improve? Can the aggregate-score change be decomposed per scenario into gate changes and weighted-component changes, including regressions as well as improvements?

  3. How does the pipeline behave with a published reward-fine-tuning recipe or a closely matched stronger optimization control? Please specify which conclusions are specific to this reward/adapter setup and which evidence supports a more general measurement claim.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 4gtK: Disjoint Test14 Evaluation, Score Split, and Wider-Adapter Control

Thank you for your thoughtful review and constructive questions. Below we address each point with new disjoint evaluations, score decompositions, and wider-adapter controls (marked in blue in the revised PDF).

Q1. Overlap with val14, disjoint Test14 evaluation, n = 1,133, and seeds

Takeaway: Although val14 shares validation logs with our fine-tuning pool, our core finding holds on the strictly log-disjoint official Test14-random split, and fixing the 1,133 to 1,118 scenario count leaves all statistical verdicts unchanged.

  • Core finding holds on unseen logs (Table 6, Sec. 4.4, App. C.12): Official Test14-random (261 scenarios) shares zero driving logs with fine-tuning. Across all 3 seeds (exact score operators), it reproduces the val14 trade-off: route progress rises (+0.004, 95% CI > 0 in every seed; 5–6 newly perfect scenarios), offset by 2–3 new zero-score at-fault collisions, leaving the mean score indistinguishable from the base BC planner. (Test14-hard shows similar two-way shifts, whereas a smaller 169-scenario disjoint validation subset did not raise progress; Tables 6 and 21.)
  • val14 overlap audit (Sec. 4.1, App. C.13): Except for DPPO, Table 2 nuPlan runs trained on 15,000 validation scenes (3 epochs, LR 1e-6; updated from the 20k/10-epoch/2e-6 setting stated in the initial text). Thus all 328 val14 logs overlap at the log level (4 scenarios share tokens, 880 share recording time, and 238 share none; Table 20), which motivated our disjoint Test14 evaluation above.
  • n = 1,133 and seeds (Table 3, Sec. 4.2): Removing 15 CSV summary rows leaves the 1,118 val14 scenarios, keeping DPM-Solver++ vs. DDIM equivalence unchanged (+0.0013 [−0.0030, +0.0055]). Main closed-loop tests (val14 and Test14 exact operators) use 3 training seeds vs. 3 BC runs; smooth-score, Wide + 4× LR (vs. 1 same-machine BC run), and Map A/C subsampling also use 3 training seeds (4 for Autoware curves; 1–2 for other ablations; App. C.14).

Q2 & W2. Continuous metrics and gate vs. component decomposition

Thank you for raising this point. We agree that the submission lacked a held-out demonstration of masked beneficial learning, and removed all "safety improvement" claims. On held-out scenes, route progress improves across all 3 seeds, whereas safety metrics (collisions and min TTC) do not; the mean score barely moves because 0/1 gate failures on a few scenarios cancel out small progress gains across dozens of others (Sec. 4.4, App. C.11, Tables 16–19):

Change vs. 3 BC runs (3 seeds)val14 (1,118 scenarios)Disjoint Test14-random (261)
Mean closed-loop score−0.0059, −0.0053, −0.0050−0.0021, −0.0001, −0.0038 (CIs cover 0)
Change via 0/1 gates−0.0062, −0.0058, −0.0055−0.0042, −0.0035, −0.0050
Change via weighted components+0.0003, +0.0006, +0.0006+0.0021, +0.0034, +0.0012
Route progress+0.0053, +0.0051, +0.0056+0.0039, +0.0038, +0.0040 (all CIs > 0)
Newly perfect (≥0.99) / lost34, 37, 40 / 6, 6, 75, 6, 5 / 3, 3, 4
Newly zero / recovered10, 11, 9 / 1, 2, 12, 2, 3 / 2, 1, 2

Min TTC shortens slightly (−0.03 to −0.04 s), at-fault collisions rise slightly (+7–8 net on val14), and peak jerk is unchanged (~−0.01 m/s³; App. C.11). Because BC 3-run noise on Test14-random (0.0064) matches the split size, we read the gate/component split on val14 only (BC spread ≤0.0005).

Q3 & W1. Wider-adapter control and scope of conclusions

Takeaway: Doubling adapted matrices (16 → 32, adding attention/conditioning), raising LR 4×, and training up to 9 epochs (3×) raises held-out reward by 25%–29% (0.442 → 0.553–0.571), yet none of the 8 control runs improves val14 (Sec. 4.2, App. C.14, Table 22).

  • Control behavior and limit: All four Wide + 4× LR runs reproduce the same trade-off (higher progress +0.0030 to +0.0108 and net perfect-scenario gains, offset by new zero scores), while single-factor changes and lowering/removing the BC penalty lose progress. Because no run beat our original peak reward (0.596), we state clearly that this tests a wider, longer setup at a comparable reward level, not stronger optimization above 0.596 (our single DPPO run stayed flat at 0.384 → 0.383, so no claim is drawn).
  • Scope (Sec. 1, 2, 5): Small component gains vs. 0/1 gate zeroing follows from the nuPlan score formula, whereas our measured sizes apply to solver-backprop LoRA recipes with a BC penalty (compatible with RL gains reported elsewhere). Our Autoware SE-LoRA study (reward-only, no closed loop) is also separated from the nuPlan closed-loop account.

We hope these experiments and clarifications address your concerns, and we welcome any further discussion.

Thanks for the reply. I still have a couple of questions.

  1. The disjoint Test14-random results support the same trade-off, and the wider-adapter control shows a similar pattern under this reward and adapter setup. Should the paper state more clearly that the negative result is specific to this recipe and optimization setup?

  2. On the disjoint split, route progress improves while collisions and minimum TTC do not, and a few new zero-score cases appear. Does this support a “beneficial learning” interpretation, or would “progress–safety trade-off” be a more accurate description?

Thanks again for the clarification.

AuthorsOpenReview ↗

Response to Reviewer 4gtK's follow-up: Recipe Scope and Progress–Safety Trade-off

Thank you for the quick follow-up. We agree on both points, and we have updated the PDF and Abstract accordingly (marked "4gtK follow-up" in blue).

1. Should the paper state more clearly that the negative result is specific to this recipe and optimization setup?

Yes. That the closed-loop score does not rise is shown for one family of recipes: low-rank adapters in the diffusion transformer trained by backpropagating the reward through the solver, with a small learning rate and a penalty keeping the plan close to the BC checkpoint. Our wider-adapter control did not reach a higher reward, so it does not extend this scope, whereas full fine-tuning or recipes that optimize the reward further may behave differently (so our findings do not conflict with gains reported by other reward fine-tuning methods). In addition to Sec. 1 and Limitations, this scope now appears directly in both the Abstract and the opening paragraph of the Conclusion (Sec. 5).

2. "Beneficial learning" or "progress–safety trade-off"?

"Progress–safety trade-off" is much more accurate, and the paper now uses this framing. With the benchmark's own score operators as the reward, the fine-tuned planner completes more of its route across seeds on both val14 and disjoint Test14-random, while at-fault collisions and minimum TTC do not improve and a few scenarios newly fall to zero through gate failures. We therefore claim no net improvement in driving: a flat or slightly lower mean score does not mean fine-tuning left the planner unchanged, because what the aggregate score masks is a repeatable progress–safety trade-off rather than a net benefit.

Updated lines in the revised PDF (marked "4gtK follow-up"):

  • Abstract: "For the recipe family we test, low-rank adapters trained through the solver, this is a trade-off between route progress and safety, not a net gain."
  • Sec. 4.4: "It trades safety for progress."
  • Sec. 5: "This is a trade-off between progress and safety, not a net gain, and it is shown for one family of recipes (see Limitations)."
Reviewer_GK2EOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies reward fine-tuning for diffusion-based autonomous driving planners, focusing on an interesting empirical mismatch: the optimized training reward can increase substantially while the aggregate closed-loop score remains almost unchanged. To investigate this, the paper introduces three main tools. First, SE-LoRA decomposes adaptation into a shared adapter and domain-specific expert adapters. Second, the authors backpropagate through the same 10-step DPM-Solver++ used at deployment, removing a potential train-test sampler mismatch. Third, PCDR is used to analyze the relation between the smooth training reward and several thresholded nuPlan scoring operators.

The main empirical findings are that sampler matching does not noticeably improve the nuPlan closed-loop score, while the shared and expert adapters appear to play different roles across domains. The paper further argues that aggregate closed-loop scores can hide small but structured behavior changes because many score components are based on discrete gates. The study is positioned as a diagnostic/measurement paper rather than a state-of-the-art planning method, which I think is the right framing. The paper also reports several negative and failed experiments quite transparently, including reward exploitation and previously invalid gradient-space analysis.

Critical strengths
  1. The main question is well motivated: if reward fine-tuning changes the model but the aggregate benchmark barely moves, it is worth understanding what actually changed. The scenario-level analysis is especially useful here. For example, the paper shows that some fine-tuned configurations simultaneously increase the number of perfect scenarios and the number of catastrophic scenarios, which is not visible from the aggregate score alone. This is more informative than simply reporting another small improvement on a single benchmark.

  2. The authors explicitly report several issues that would often be buried or omitted: a map-projection bug affecting Map C, possible overlap between the nuPlan fine-tuning pool and val14, a multi-seed rerun that exploited the reward, and an earlier gradient analysis that was invalid because independently initialized adapters were compared in inconsistent coordinate systems. I consider this a real strength because it makes it much easier to judge which conclusions are reliable and which remain tentative.

Critical weaknesses
  1. The reported 2.5× increase in training reward is shown on the Autoware experiments, whereas the flat closed-loop result is primarily demonstrated on nuPlan val14. The paper therefore combines two observations from different stacks into one central narrative. On nuPlan, the best fine-tuned configuration reaches 0.8975 ± 0.0013 versus 0.8973 for the released BC model, while the other variants perform worse, but corresponding reward improvement curves for these same runs are not clearly reported. This matters because the main conclusion is much stronger if the same model is shown to substantially optimize its reward while failing to improve closed-loop behavior. I would like to see the before/after training or held-out reward for the nuPlan configurations in Table 2, paired directly with their closed-loop results.

  2. The paper argues that small domains rely more on the shared adapter, while larger domains rely increasingly on their expert adapters. However, the four Autoware domains differ not only in dataset size, but also in map type, geometry, and reward weights. Map C, for example, is simultaneously the smallest dataset, a test-track environment, and uses a different reward profile. The current evidence therefore shows a correlation, but not clearly that dataset size is the reason for the different adapter behavior. A much stronger test would be to subsample a single domain at several training-set sizes while holding the environment, reward, architecture, and optimization setup fixed.

Questions for authors
  1. Can the authors report the reward improvement for the same nuPlan checkpoints evaluated in Table 2? In particular, I would like to know whether the smooth-score and DPM-Solver++ fine-tuned models substantially improve their training or held-out reward while the val14 score remains unchanged. This would directly test the paper's central reward/closed-loop mismatch within one experimental setting.

  2. Can the authors isolate the effect of training-set size on shared versus expert reliance? For example, could Map A or another single domain be subsampled to several dataset sizes and retrained under the same reward and environment? This would make the current data-size interpretation substantially more convincing.

1 comment
AuthorsOpenReview ↗

Response to Reviewer GK2E: Same-Stack nuPlan Rewards and Controlled Single-Map Subsampling

Q1 & W1. Do the nuPlan checkpoints in Table 2 raise their reward while val14 closed-loop scores stay flat?

Yes. On the same nuPlan checkpoints, three training rewards rise by 30% to 60% on held-out scenes (and by 16% to 31% on 426 scenes from disjoint logs), while their val14 closed-loop scores do not rise.

For each row with a reward entry, we scored the checkpoint with its training reward before and after fine-tuning on 1,500 held-out scenes from the fine-tuning logs, and compared val14 runs scenario-by-scenario against the released BC planner (row 1 shows the run-to-run spread among 3 BC simulations; brackets are 95% bootstrap CIs; perfect means ≥0.99):

Training rewardHeld-out reward (before → after)val14 minus BCPerfect (new / lost)Zero (new / lost)
BC checkpoint (each run vs. other two)−0.0004 to +0.00050 to 2 / 1 to 30 to 1 / 0 to 1
Soft safety margin, DPM-Solver++0.384 → 0.538 (+40%)−0.0131 [−0.0207, −0.0058]14 / 2019 / 3
Soft safety margin, DDIM0.390 → 0.540 (+38%)−0.0144 [−0.0222, −0.0070]15 / 2919 / 3
Exact score operators (3 seeds)0.442 → 0.596, 0.582, 0.574−0.0059, −0.0053, −0.005034, 37, 40 / 6, 6, 710, 11, 9 / 1, 2, 1
Five score operators0.507 → 0.811 (+60%)−0.0084 [−0.0146, −0.0028]50 / 1114 / 1
Smooth copy of score (3 seeds)0.315 → 0.319 (seed 42)−0.0009, −0.0008, −0.00168, 6, 10 / 13, 4, 56, 3, 4 / 3, 1, 2
DPPO (1 run)0.384 → 0.383−0.0019 [−0.0050, +0.0010]4 / 42 / 1
  • The two models you asked about: DPM-Solver++ (soft safety margin) raises held-out reward by 40% while val14 falls by 0.0131. Smooth-score barely moves its reward (0.315 → 0.319) and every seed's CI includes zero (0.8975 ± 0.0013 was the 3-seed mean; Table 2 now lists per-seed paired differences, where seed 42 re-run on a second machine scores 0.8956 vs. 0.8965 BC; Table 25 lists an earlier simulation of the smooth copy at 0.899 vs. 0.897).
  • Why the mean moves so little: With exact score operators (exact copies of nuPlan's collision, progress, and direction operators; all CIs below 0 on val14), 34 to 40 scenarios per seed newly reach ≥0.99 nearly all via route progress (32 shared across all 3 seeds), while 9 to 11 newly fall to 0 via collisions or leaving the drivable area (9 shared across seeds). On disjoint Test14-random (Table 6), route progress rises in every seed (CI > 0) while the mean is indistinguishable from BC.
  • One stack for closed loop: All closed-loop claims now come from nuPlan (Table 2, Sec. 4.2, 4.4). Autoware serves the reward-based adapter measurements below (in closed-loop Autoware simulation, App. C.2, fine-tuned checkpoints finish at most 1 of 32 runs vs. 24 to 27 for BC, so no closed-loop claim is drawn from Autoware).

Q2 & W2. Once map and reward are held fixed, is training amount the reason for the adapter behavior?

With map and reward held fixed, on both maps that we varied, the Frobenius norm of the map's per-domain (expert) adapter follows the number of updates it receives rather than the number of distinct scenes.

We subsampled Map A alone from 2,663 scenes to 172 (Map C's size) with the other three maps, rewards, recipe, and 297 Map A validation scenes held fixed across 3 seeds per arm, plus a 5th arm repeating the 172 scenes to match full updates (Table 4, Sec. 4.3, App. C.8; criteria fixed before training):

Map A training scenesMap A expert normMap C expert norm (same runs)Share of Map A reward gain from expert
1720.142 ± 0.0050.214 ± 0.0080.08 ± 0.02
5000.285 ± 0.0070.199 ± 0.0180.12 ± 0.02
1,0000.436 ± 0.0060.185 ± 0.0070.22 ± 0.10
2,663 (all scenes)0.653 ± 0.0210.161 ± 0.0120.26 ± 0.06
172 (repeated to full updates)0.648 ± 0.0120.162 ± 0.0130.25 ± 0.05
  • Updates vs. scene count & reverse Map C test: Repeating 172 scenes matches full data in both norm (0.648) and reward share (0.25). Conversely, repeating Map C's 172 scenes to match Map A's updates raises Map C's norm from 0.15 to 0.17 up to 0.72 to 0.73 and reward share from ≤0.10 to ≥0.25 in all 3 seeds.
  • Reading and scope: Our pre-fixed criterion also required reward share to grow at every step; in one seed it is 0.33 at 1,000 scenes and 0.32 at 2,663, so the paper bases the statement on adapter norm and reports share beside it. At 172 scenes, Map A (0.142) ends below Map C (0.214), so the map also contributes; in each experiment the adapter of the unchanged map shifts by ~0.05 (reported without explanation). Both experiments are measured on Autoware reward without closed loop.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.