acceptodds
Under review as a conference paper at ICLR 2027

Scalable Rollout Generation for Generative Driving Simulators

Abstract

Generative simulators are emerging as a promising foundation for post-training of autonomous driving policies. However, scaling training in these environments is limited by the cost of sensor simulation and the sequential dependency between observation rendering and policy inference. The simulator waits for the policy to predict the next ego pose, while the policy waits for the resulting observation before replanning. This ordering introduces idle time and limits rollout throughput. Inspired by speculative decoding, we introduce Speculative Rendering, a rollout-generation framework that speculates future simulation states ahead of planner execution. Our approach renders future observations in parallel with the planner, and employs a verifier to determine whether the speculated state remains sufficiently close to the realized state. When the divergence exceeds an acceptance threshold, the speculated renders are discarded and the simulator state is resynchronized with the target planner. We evaluate Speculative Rendering for GRPO post-training of AutoVLA using WorldEngine, a generative driving simulator. Speculative Rendering reduces mean simulation-step latency from to  ms and lowers rollout-generation cost from to H100 GPU-hours, a reduction. The resulting policy trained on speculative rollouts improves PDMS on the challenging navtest-failure benchmark by and remain within of the performance obtained using standard sequential rendering.

Primary area:
applications to robotics, autonomy, planning
Keywords:
generative simulators, speculative rendering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_cPNROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Speculative Rendering: a method motivated from speculate decoding in LLMs, to pre-render likely future simulation states before the driving policy/planner has executed that far. The rationale for speculative rendering is to reduce idle GPU time, by proactively rendering multiple likely-futures states in parallel as a batch, rather than the slower vanilla process of sequentially iterating between inferring policy actions and rendering the next simulation state. The trick is to understand whether the future renderings are “close enough” to be valid, or should they be discarded and when should the simulator indeed need a “true” rendering at some future time step.

While I really appreciate how well this problem was motivated, I have concerns about how much value the proposed solution provides. I've also left some questions too.

Critical strengths
  1. Well motivated: sensor simulation for closed-loop evaluation and/or fine-tuning of end-to-end autonomous stacks can be expensive and slow; especially when there’s communication delays between the stack/policy and the renderer that iteratively wait for each other before stepping, leading to inefficient use of compute. Paper seeks to reduce the GPU’s idle time and increase the parallelization of compute.
  2. Well written: the language was clear and the points logically presented. Figures are also clear, such as Figure 1’s simplicity, which is nice.
  3. Novel solution: I had not seen something like this before.
Critical weaknesses

Large weaknesses:

  1. While motivated from speculative decoding (positive point), the analogy of a draft-and-target models here seems weak given their often non-overlapping support, i.e. the (deterministic) renderer from a slightly shifted/erroneous simulation states will often be such that there is no change of them drafting a render that the “target model” could have rendered (if I understand correctly), so any analogy of rejection sampling here seems difficult (unless perhaps the state predictions themselves were probabilistic?), i.e. it would always “reject” ?
  2. Sanctity check: it looks like in Table 1 we can reduce the GPU hours by half, without compromising too much on PDMS. How would this all compare to a simple baseline, such as: predict actions at 0.5s, render at 1s (already). Figure 8b already shows ~80% re-use at k=1, so this simple baseline might work almost just as well?

Small weaknesses:

  • Why not have “Speculative Rendering” or “SpecRender” in the title? (preference for “Speculative Rendering”). Possibly too late for ICLR, but if possible, perhaps worth consideration.
  • Use a consistent method name. Both “Speculative Rendering” and “SpecRender” are used frequently, interchangeably. Pick one?
  • Line 090: presenting example benefits without associated costs like “latency decreases from 364 to 138 ms” and “cost decreases from 840 to 488 H100 GPU-hours” is a bit meaningless given the aforementioned inherent trade off mentioned on Line 086. Perhaps pair it with the modest performance decrease (71.33 versus 72.13 PDMS) so the readers can see the trade off of using where the speculative renderings which contained small errors. Or perhaps there is a single metric the author could show, something like model performance per wall clock time of training (which presumably is better than vanilla training, given the commute usage increase >> model performance drop), for a given threshold chosen.
  • Line 198: For equation (4), do the authors mean “s_{t’}” or “s_’{t}” ?
  • Line 204: “predicts N future poses”: can the authors please clarify what N is. Is this N possible ego states at next time step (time t) given prior observations, or is N time steps into the future, predicted from t? Edit: I see later on Line 215 “K<=N” the answer is: steps into the future from t.
  • Line 206. Why “a” in (5) but “x” in (2), pick one convention?
  • Line 206: Consider using subscript “n” instead of “j” as a subscript over “{1,..., N]”. This is also confusing in equation 6 where j is re-used for {1,…, K}. Perhaps consider not telling the reader about the N future poses, since the superfluous N-K state predictions don’t matter; instead mention K predicts to match K renderings, and change “j” to “k”?
  • Line 220: notation of using “t” in both the subscript and superscript seems redundant. Also the “t+j” of (6) doesn’t match the “t,j” of (5), pick one convention and stick with it? If the authors like the superscript “(t)” convention, then the subscript can simply be “j”. Whichever chosen, is Figure 2 yet another disagreement by using \tau ?
  • Line 665: use {} to preserve capitalization for names in bibtex, e.g. {Gaussians} and {Gaussian}
Questions for authors
  1. Is there a way a user could understand how to set thresholds in (7) to understand (in advance) a trade off of policy quality with rendering error?
  2. If rendering is expensive, but simulator state errors are still small (measured by equation 9), could we take advantage of (cheaper) depth-based reprojection rendering given (1) speculative/cached renderings and (2) knowledge of the simulation state error (still within threshold limits)?
  3. Line 256: “Checking reactive actors in addition to the ego constrains position errors in the surrounding traffic, but can reduce the acceptance rate”: is the rendering cost for other agents much less than the surrounding (Gaussian splatt) scene? If so, is there a potentially more complex variant of this method which can leave draft-render the scene, but post-hoc render other agents efficiently?
  4. Could errors accumulate over time in unbounded ways? I’m unsure, but thinking about equation (9) which verifies a bound on the simulator state errors between (1) plans conditioned from current time, (2) re-plans conditioned from future speculative renderings; but never against (3) re-plans conditioned on future real renderings, had we played the simulator out normally. While computing (3) would defeat the purpose of this paper, are there any situations where (1) and (2) could stronger agree over long time horizons (i.e. always below verification threshold), but would acclimate error with (3) (had it been computed) unbeknownst to the user?
2 comments
AuthorsOpenReview ↗

Response to Reviewer cPNR (1/2)

We thank Reviewer cPNR for their time and constructive feedback. We address the concerns below.

  1. Speculative decoding analogy
    We thank the reviewer for the comment. As stated in L17 and L79, our method is inspired by the drafting and verification components of speculative decoding and we do not claim its distribution preservation guarantees. As the reviewer notes, with a deterministic renderer, rejection sampling would reject a draft whenever it differs from the target rendering. Because our verifier allows small drifts, speculative observations may shift subsequent policy predictions, producing rollouts slightly off-policy relative to standard rendering. We assume that a strict verifier keeps these differences small enough to preserve the RL training signal, as supported by Tab. 1 and Fig. 5. We will clarify this distinction.
    Exact rejection sampling would require the entire framework to support probabilistic ego and traffic states, which is uncommon in closed-loop simulation. It would therefore require probabilistic planners and traffic controllers producing trajectory distributions. Instead, SpecRender is designed as a plug-and-play technique compatible with off-the-shelf planners such as AutoVLA and VADv2 (see upcoming response to R-e1oT).
    A simpler way to enable rejection sampling would be to treat point-state estimates as centers of Gaussian distributions, with a diagonal scale hyperparameter λ\lambda controlling support overlap. This would serve a similar role to the acceptance thresholds ϵpos\epsilon_{pos} and ϵact\epsilon_{act}. We will consider this probabilistic extension in future work.

  2. Baseline experiment
    Following the reviewer’s suggestion, we evaluate a baseline that renders every 11s while replanning every 0.50.5s using the latest available observation. This halves the number of rendering cycles but retains the sequential dependency. At t=0.5t=0.5 s, the planner reuses the observation from t=0t=0 while the simulator waits for the next action. At t=1t=1 s, the planner stalls while the simulator renders the new observation, which is reused at t=1.5t=1.5 s. This baseline reduces simulation-step latency from 364.48364.48 ms to 264.12264.12 ms, compared with 138.91138.91 ms for SpecRender. The k=1k=1 reuse refers to verified observations rendered in advance for the next simulation step, which explains the further latency reduction.

  3. Setting the threshold
    We evaluate candidate thresholds on held out scenes by measuring 1) L2 distance between speculated and target poses 2) PDMS degradation with respect to non-speculated trajectories and 3) latency. We then select the L2 distance that best compromises between latency and PDMS degradation. We further validate the hyperparameters by evaluating the policy’s open-loop L2 error, which in our case was approximately 0.50.5m at 2 sec lookahead. We will add a description of these procedures in the Appendix.

  4. Depth Reprojection
    Thank you for this interesting suggestion. We could render and cache depth maps alongside RGB images, then resample and “correct” the RGB images based on the difference between drafted and realized camera poses through depth reprojection. This could increase draft acceptance and avoid unnecessary asynchronous re-renders.
    However, it would be beneficial only if the overhead of depth rendering, caching, and resampling were lower than that of a fresh render. Moreover, even a slight change in viewpoint might reveal content hidden in the original render, requiring to integrate the full appearance via other cached frames or additional targeted rasterizations. Finally, camera reprojection would only fix errors in speculated ego state, but not in actor motion, that would require to be handled separately. The idea is sound and it opens several interesting research questions, and we regard it as a future potential extension of SpecRender.

  5. Selective Actor Rendering
    Indeed, 3DGS rasterization cost scales linearly with the number of input Gaussians, which is typically much higher for the background than for other agents. We could therefore distinguish rejections caused by incorrectly speculated ego states from those caused by incorrectly speculated traffic. These cases could then be handled separately, e.g. by depth-reprojecting the background (as above), or by re-rendering only dynamic agents. We will explore these further optimizations in future work.

  6. Minor suggestions on notations and title
    Thank you for these suggestions. We will integrate the changes in the final version.

AuthorsOpenReview ↗

Response to Reviewer cPNR (2/2)

  1. Unbounded error accumulation
    Thank you for raising the question of cumulative rollout drift. We agree that Eq. (9) controls local draft-position error but does not bound trajectory separation from standard rendering over arbitrarily long horizons. For example, at an intersection, a small difference between an accepted cached image and a fresh rendering of the realized state could change the policy’s decision to turn, after which subsequent drafts might pass verification along a different trajectory.
    Because AutoVLA predictions are stochastic, evaluating such counterfactuals is challenging. Fig. 5 shows closely aligned trajectories and similar reward distributions, while Appendix A.4 intentionally presents failure cases in which small differences are amplified through replanning and reactive traffic. Error accumulation is therefore possible, but our focus is its practical effect on the RL training signal, which appears limited: training with SpecRender achieves 71.3371.33 PDMS under standard-rendering evaluation, compared with 72.1372.13 for standard-rendering training.
Reviewer_FoxxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper speeds up closed-loop rollout collection in a generative driving simulator by rendering future observations before the planner has committed to later poses. AutoVLA’s predicted trajectory suffix, plus an IDM rollout of nearby actors, is rendered ahead up to K=3 and cached. A kinematic check then decides whether to reuse that frame or re-render. In World Engine, step latency drops from 364 ms to 139 ms, and GRPO rollout cost from 840 to 488 H100 GPU-hours. The policy trained on these rollouts reaches 71.33 PDMS on navtest-failure, against 72.13 for standard rendering and 58.67 for the supervised checkpoint.

Critical strengths

The paper closes the loop instead of stopping at a latency microbenchmark. Rollouts are generated in World Engine with reactive IDM traffic, scored with the PDM reward, and used for GRPO on AutoVLA: 16 candidates per scene, 20 frames over 12 seconds. The trained policy is then evaluated in reactive closed-loop simulation on navtest-failure. SpecRender plus GRPO reaches 71.33 PDMS, SR 76.01, and EP 69.31, against 72.13 / 75.96 / 68.39 for the same GRPO with sequential rendering, at 488 rather than 840 H100 GPU-hours. The supervised starting point is 58.67. At a matched budget, keeping fewer scenes, fewer candidates, or a shorter horizon lands between 63 and 68 PDMS. That is a complete pipeline result: the proposed generator is actually used to train a policy, and the policy is tested closed-loop. It is what makes the empirical claim solid enough to take seriously.

Critical weaknesses

1、The speculation is multi-step, and the error control is still a per-step gate. A draft at depth j is accepted when the ego position and the nearby actors sit inside a tolerance at that step. Heading is left ungated. The accepted frame is fed to the policy, the policy replans, and the next draft is checked against this new state. A gap that passes ε at every step can still accumulate: each inexact observation moves the next plan, reactive traffic answers a different ego, and a later discrete event flips the outcome. Appendix A.4 and Figure 7 show four matched rollouts that start close and then leave the drivable area, split at a junction, or collide. Figure 5 and the 0.80 PDMS gap, from a single run, do not measure that drift along the K-step chain. The regret-based rule, which does keep a budget on pose error, appears only in the speed plot. The GRPO runs use the per-step KERV gate. The closed-loop number shows that training finished. It does not show that the multi-step drafts stayed faithful while it finished.

2、The per-step gap is also hard to read. Under KERV the paper quotes hit rates of 96%, 78%, and 54% at depths 1–3, so almost half of the depth-3 drafts miss. Appendix A.3 says KERV accepts every draft in the profile behind the 364 ms to 139 ms number. Section 4.1 sets the position base to 0.5 m; Appendix A.1 uses 0.25 m, which is the value that gives the stated 0.125–0.5 m range. Figure 4 rejects frames at a fixed 0.25 m. Yaw is infinite, and actors beyond 40 m are not checked, so a passing position gap is not the observation the policy consumes.

4、The methodological novelty could be better articulated.

Questions for authors

1、For rollouts that accept a chain of drafts, what is the distribution of cumulative ego position and heading error over the K steps, and how often does a chain of individually accepted steps end in a different collision or drivable-area outcome? The final PDMS after GRPO does not answer this. 2、Which gap produces the 54% depth-3 hit rate, and which gap produces the profile in which KERV accepts every draft? 3、The regret-based verifier already accumulates pose error, but it is not used for the GRPO runs. If that budget is enforced while collecting the training rollouts, what happens to GPU-hours and to navtest-failure PDMS?

1 comment
AuthorsOpenReview ↗

Response to Reviewer Foxx

We thank Reviewer Foxx for their time and constructive feedback. We address the concerns below.

  1. Rollout drift
    We agree that a per-step verifier can accept drafts that gradually drift from the standard-rendering trajectory, as also discussed in our response to R-cPNR under 7. Unbounded error accumulation. However, in our experience this is rare, and significant failures as the ones presented in Fig. 7 occur occasionally. To quantify this drift, we measured the position error and heading error between the SpecRender training rollouts used in Table 1 and the standard-rendering AutoVLA baseline, finding it below 1.431.43m and 12.67∘12.67^\circ in 95%95\% of comparisons. In our training rollouts, the highest-reward candidates within each group had no collision or drivable-area penalty in 91.3%91.3\% of training group, while the rollouts between SpecRender and standard rendering differed in collision outcomes in 2.08%2.08\% of comparisons and drivable-area outcomes in 6.25%6.25\% . Additionally, the small PDMS difference after training, 71.3371.33 vs. 72.1372.13 with both policies evaluated using standard rendering, suggests that the observed drift has a limited effect on the training result.

  2. Threshold value and depth-3 hit rate
    Thanks for identifying these inconsistencies. The base ego-position tolerance is 0.5m0.5m and scales with drafted ego speed according to Eq. (21). With the scale clipped to [0.5,2.0][0.5,2.0], the effective tolerance ranges from 0.250.25 to 1.0m1.0m. We will correct Appendix A.1 and A.2 and clarify that Figure 4 illustrates reuse and fallback with a fixed threshold.
    Concerning the hit-rate at different depths, Fig. 3(a) measures the probability that drafts at exact depths are accepted: for instance, a draft produced for k=3k=3 is within the tolerance 54% of the time. Fig 3(b) measures something different: at any given simulation step, KERV verification always guarantees an accepted draft (either coming from t−1,t−2,t−3t-1,t-2,t-3). This number is higher here because it suffices that any draft produced for the current timestep passes verification, regardless of the time at which it was produced.

  3. Regret-based verifier
    The regret-based verifier is labeled ASD in Fig. 3(b), where it has 13.4% slower speedup than KERV. Using the exact same settings, we observe that the overall training rollout for regret based verfier is 537 GPU hours vs. 488 GPU hours with KERV, and ASD achieves a PDMS of 69.91 vs 71.33 and as a result, we use KERV as our default verifier.

Reviewer_e1oTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces SpecRender, a framework for accelerating closed-loop rollout generation in driving simulators. It uses the driving policy’s predicted trajectory and reactive actor dynamics to render future observations asynchronously during policy inference. A state-based verifier determines whether cached observations can be reused; otherwise, the simulator renders observations from the realized state.

The framework is evaluated using AutoVLA in World Engine on NAVSIM-v1 scenes, including downstream GRPO post-training. The reported rollout-generation cost decreases from 840 to 488 H100 GPU-hours, a reduction of approximately 42%. The resulting policy achieves 71.33 PDMS on navtest-failure, compared with 72.13 using standard rendering, and outperforms the evaluated alternatives with similar rollout-generation budgets.

Critical strengths
  1. A practical mechanism for reducing rollout-generation latency. SpecRender reuses future trajectory predictions already produced by the driving policy to prepare observations asynchronously, without requiring an additional learned drafter. It incorporates reactive actor prediction, state-based verification, and fallback rendering while retaining policy replanning at every step. This provides a concrete way to exploit otherwise unused trajectory predictions and overlap rendering with policy inference, addressing an important bottleneck in closed-loop simulation.

  2. Validation through downstream policy training and comparable-budget baselines. The evaluation goes beyond rendering latency to test whether accelerated rollouts remain useful for GRPO post-training. The complete system reduces reported rollout-generation cost from 840 to 488 H100 GPU-hours, while achieving 71.33 PDMS compared with 72.13 under standard rendering. It also outperforms the evaluated alternatives at similar rollout-generation budgets. These results provide encouraging evidence that the approach can improve practical training efficiency while retaining much of the benefit of standard-rendering post-training.

Critical weaknesses
  1. The contribution of speculative rendering to the reported acceleration is not sufficiently isolated. Appendix A.3 describes additional GPU-based image remapping, JPEG encoding, and parallel frame-writing optimizations, but it is unclear whether the standard-rendering baseline uses the same optimizations. Without a controlled comparison that shares these optimizations, the reported speedup cannot be cleanly attributed to speculative rendering. Furthermore, no K=1 versus K=3 runtime ablation is provided to establish the incremental benefit of multi-step lookahead. These omissions leave the contribution of the proposed mechanism insufficiently demonstrated, despite the useful acceleration achieved by the overall implementation.

  2. The evaluation provides limited evidence for the reliability and generality of the speed–quality trade-off. Experiments are restricted to AutoVLA with World Engine, although the method’s effectiveness depends on trajectory stability under replanning and the relative costs of policy inference and rendering. It remains unclear whether comparable gains would hold for other policies or simulator configurations. Within the evaluated setting, Table 1 reports a 0.80-point PDMS decrease without uncertainty estimates. The coarse reward histograms and selected trajectories in Figure 5 do not establish preservation of the training signal, while Appendix A.4 shows divergent collision or off-road outcomes without quantifying their frequency. Together, these limitations weaken confidence that the reported trade-off is robust beyond the demonstrated configuration.

Questions for authors

Attribution of the reported speedup. Appendix A.3 attributes part of the improvement to GPU-based image remapping, JPEG encoding, and parallel frame writing. Does the standard-rendering baseline include these same optimizations? If not, could you report the end-to-end speedup when both methods use them under the same total hardware budget? Please clarify whether the reported GPU-hour costs include all resources used by background rendering workers. This would establish how much of the gain is attributable to speculative rendering.

Execution schedule and the contribution of lookahead. Section 3.2 defines depth one as the pose selected for immediate execution, while Appendix A.5 reports that 81.8% of reused speculative inputs come from this depth. Could you provide a timestep-indexed execution timeline showing when this pose becomes available, when its rendering starts, and which policy inference overlaps with that rendering? Please clarify whether “depth one” has the same meaning in these sections. Once this is established, a controlled K=1 versus K=3 runtime comparison would help isolate the benefit of rendering beyond the immediately executed pose.

1 comment
AuthorsOpenReview ↗

Response to Reviewer e1oT

We thank Reviewer e1oT for their time and constructive feedback. We address the concerns below.

  1. Latency of baseline vs SpecRender

    We confirm that all baseline latencies in our paper are measured after applying the same GPU image-remapping, JPEG-encoding, and parallel frame-writing optimizations as described in Appendix A.3. All methods use identical hardware and worker counts. Therefore, the reported 364.48 ms to 138.91 ms latency reduction is fully attributable to SpecRender. The reported GPU-hour costs include all eight H100 workers, including the resources used for background rendering. We will clarify these points to avoid confusion.

  2. Latency at K={1,2,3}K = \{1,2,3\}

    Thanks for requesting this ablation. Following Appendix A.3, we report the latency for K=1K=1 and K=2K=2 and compare it with the default K=3K=3 using the exact same settings and hyperparameters.

    Component (ms)MTGSK=1K=1K=2K=2K=3K=3
    engine.before_step0.060.050.070.06
    engine.physics_step15.5411.8912.2412.18
    engine.after_step67.5566.1465.2764.31
    sensor.get_observations196.1579.4369.9162.36
      render.total103.6258.1048.3341.38
      io.frame_write88.6721.7221.5818.58
    Total step364.48157.51147.49138.91

    We observe that increasing the lookahead from K=1K=1 to K=3K=3 reduces simulator-step latency from 157.51157.51 ms to 138.91138.91 ms, an additional 11.811.8% reduction. Although depth-one drafts supply 81.881.8% of speculative planner inputs, completed deeper drafts can serve a target step when its newer draft is not yet ready. This explains the role of multi-step lookahead in improving observation availability. We will add this ablation to the Appendix.

  3. SpecRender with an additional planner

    Following the reviewer's suggestion, we implemented SpecRender using VADv2 as the planner. VADv2 predicts a 40-pose ego trajectory at 0.10.1 s intervals, which we sample at the simulator's 0.50.5 s interval to draft the next K=3K=3 observations while retaining the same KERV verification and fallback rendering. With standard rendering, post-RLFT VADv2 achieves 67.5667.56 PDMS and requires 1,0721{,}072 H100 GPU-hours. VADv2 trained using SpecRender rollouts achieves 67.3567.35 PDMS, while rollout generation requires 651651 H100 GPU-hours. These results suggest that SpecRender generalizes to planners with different architectures and replanning behavior.

  4. Uncertainity estimates
    Following the reviewer's suggestion, we repeat GRPO training using AutoVLA rollouts with three independent seeds. Under standard-rendering evaluation, SpecRender achieves 71.42±0.4971.42 \pm 0.49 PDMS showing that variance is small. We also find no significant difference in candidate-reward distributions between the three SpecRender seeds. We will include these uncertainty estimates in Table 1.

  5. Timestep-indexed execution timeline
    Thank you for this suggestion. We will add the execution plot in the Appendix.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Follow-up on our rebuttal

Dear Reviewers,

Thank you again for your thoughtful feedback on our work. We hope our rebuttal has addressed your comments about the paper, and we would appreciate hearing whether any concerns are left. We welcome further considerations and are happy to clarify any remaining points before the discussion period ends.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.