acceptodds
Under review as a conference paper at ICLR 2027

PercepDraft for Efficient Generative Perception

Abstract

Generative perception models express different visual perception tasks as one generation process, in which sparse tasks such as detection are decoded autoregressively into structured symbol sequences and dense tasks such as segmentation, depth and surface normals are produced as pixel-level perception maps through iterative denoising. Both generation processes call the full backbone repeatedly, which makes inference expensive. We find that the structural regularity inherent in perception outputs makes their generation predictable from step to step: the serialization pattern of a structured sequence is largely fixed, and the global spatial layout of a dense perception map is essentially formed early in denoising. Based on this observation we propose , which keeps the pretrained backbone frozen and introduces two lightweight drafters, a token drafter and a trajectory drafter. They respectively propose several future tokens, or future denoising velocities with per-offset error estimates used to select a trusted horizon. The backbone then verifies the proposals in one call, so that fewer backbone calls are needed at inference. Experiments confirm the effectiveness of across multiple generative perception models. On , it achieves end-to-end speedups of up to 1.6, 2.0, 6.3, and 5.4 on detection, panoptic segmentation, depth estimation, and surface normal estimation, respectively, with limited changes in task performance.

Primary area:
generative models
Keywords:
multimodal models, adaptive computation, efficient inference,Generative Perception

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_e7mrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

PercepDraft accelerates generative visual perception by exploiting predictability along autoregressive and denoising trajectories while keeping the backbone frozen. For structured text outputs, it uses draft heads and backbone prefix verification. For dense outputs, a lightweight spatial network predicts future velocity offsets and per-horizon errors. The paper evaluates detection, panoptic segmentation, depth, and surface normals across SenseNova-Vision and additional task-compatible backbones. The results show that the proposed methods substantially reduce inference time compared with the original decoding procedures, while largely maintaining task performance.

Critical strengths
  1. Substantial practical acceleration, particularly for dense perception. On SenseNova-Vision, depth latency decreases from 8.66 to 1.38 seconds and normal-estimation latency from 8.70 to 1.61 seconds, with comparable aggregate accuracy. Panoptic latency decreases from 12.1 to 5.91 seconds while PQ changes from 51.4 to 51.3. These measured wall-clock improvements are meaningful, and validation on additional backbones suggests that the approach is not restricted to one output interface.

  2. A lightweight and practically motivated adaptation strategy. The method preserves backbone weights and introduces small auxiliary predictors rather than retraining the full perception model. The dense design addresses an important task-dependent trade-off: geometry tolerates aggressive approximation, whereas panoptic segmentation is more sensitive. The error-based horizon selection and endpoint check provide a plausible mechanism for adapting computation to this difference. The paper also reports failure cases and explicitly acknowledges that endpoint verification does not certify the entire trajectory.

Critical weaknesses

The incremental contribution of the proposed components is not sufficiently isolated from simpler alternatives. Constant-velocity drafting already achieves 3.84 depth AbsRel at 16.2 calls, versus 3.85 at 13.7 calls for the complete separately trained variant (Table 7), indicating that much of the headline depth acceleration is available without learned velocity prediction. Likewise, restricted-vocabulary parallel token heads reach 3.46 tokens/call versus 3.54 for PercepDraft at identical F1 (Table 8). These comparisons do not negate the method’s usefulness, but they leave its additional complexity and methodological contribution insufficiently justified.

Questions for authors
  1. Can you report matched-quality wall-clock comparisons using batched FlowCast verification? Please distinguish results from faithful implementations from those of the simplified adaptations
  2. Can you provide quality–latency comparisons for the velocity-source/horizon combinations at identical tolerances and comparable training settings? For the token branch, what wall-clock benefit remains over restricted-vocabulary parallel heads under the same verifier? How much dense acceleration remains relative to tuned reduced-step schedules, including the eight-step Marigold baseline already reported?
5 comments

Response to Reviewer e7mr (1/3)

We sincerely thank the reviewer for the detailed review and for recognizing the substantial wall-clock acceleration on dense perception, the lightweight adaptation that keeps the backbone frozen, and the error-based horizon selection with endpoint verification that adapts computation to the different tolerance of geometry and panoptic segmentation. All changes are marked in blue in the revision.

[W1] Isolating the contribution of each component from simpler alternatives.

[A1] Thank you for this careful observation. We reran the comparison under one unified setting, so that all rows now share the same tolerance and horizon rule (in Appendix C.2.2 the rows with and without the trusted horizon used different tolerances).

(1) The 2×2 comparison under a unified setting. All four combinations use the per-task tolerances of Table 1 (0.4 for geometry, 0.2 for panoptic segmentation) and the same images. Both fixed-horizon combinations draft 8 steps per segment, both trajectory-drafter combinations use the same trajectory drafter, and the trusted horizon of the constant-velocity row comes from an error head trained with the same data and number of steps. Calls are backbone evaluations per image (98 for the reference), and speedups are wall-clock ratios to the reference on the same GPU.

Velocity sourceHorizon ruleDepth AbsRel ↓Calls ↓Normals ↓Calls ↓Panoptic PQ ↑Calls ↓
Constant velocityFixed, 8 steps3.9320.013.8230.051.755.0
Trajectory drafterFixed, 8 steps3.9019.613.7929.651.451.8
Constant velocityTrusted horizon3.9513.613.7915.850.941.7
Trajectory drafterTrusted horizon (PercepDraft)3.8713.413.7715.851.338.7

The speedups (depth / normals / panoptic) of the four rows are 4.5× / 3.2× / 1.5×, 4.6× / 3.3× / 1.6×, 6.2× / 5.4× / 1.9× and 6.3× / 5.4× / 2.0×.

The trusted horizon provides most of the reduction in calls (with the trajectory drafter, from 19.6/29.6/51.8 to 13.4/15.8/38.7 at essentially unchanged quality), and the trajectory drafter's velocities further lower the error and the calls (depth AbsRel 3.95 to 3.87; panoptic 50.9 to 51.3 PQ with 41.7 to 38.7 calls). Constant velocity with a fixed horizon, the alternative you mention, needs clearly more calls at the same tolerance (20.0/30.0/55.0 vs. 13.4/15.8/38.7), while PercepDraft also has lower depth error (3.87 vs. 3.93) and a higher speedup (6.3× vs. 4.5×). These unified results replace the corresponding rows of Table 7 and Table 3, with the tolerance and horizon rule stated.

(2) Landing-point verification makes even simple drafts effective. The constant-velocity row already benefits from PercepDraft's landing-point verification. The depth trajectory is nearly straight in its middle part (2.8% to 3.9% velocity change between adjacent steps, Table 11), so most constant-velocity drafts pass verification; the same training-free drafts also reduce the 16-step Pixel-Perfect Depth sampler from 16 to 5 calls with lower error. On top of this, the trusted horizon decides how far to advance from the predicted error of each segment, so one tolerance per task suffices, and the trajectory drafter's velocities make the landing point more accurate. Section 4.6 of the revision states the role of each component accordingly.

(3) Token branch. The restricted record vocabulary is part of PercepDraft's token drafter (Section 3.4), and the restricted-vocabulary parallel heads in Table 8 are an ablation of our own design without chained prediction heads, not an existing alternative. With the same verifier and runtime, the per-image latency (s) of the three token-drafting heads is as follows.

Token headsLVIS ↓VisDrone ↓Dense200 ↓
Medusa (full-vocabulary parallel heads)1.508.3328.7
Parallel heads, restricted vocabulary (ablation in Table 8)1.296.8223.4
PercepDraft (restricted vocabulary + chained heads)1.256.6523.5

PercepDraft's token branch is 1.20–1.25× faster than the existing Medusa heads. Restricting each head to the 1718 token ids used by perception records accounts for most of this gain; chaining the heads, so that each head conditions on the predictions of the previous ones, raises the committed tokens per call from 3.46 to 3.54 and gives a further 3% and 2.5% on LVIS and VisDrone. The two are tied on Dense200, where the accepted length is limited by the last coordinate digit, which both kinds of heads find equally hard to predict.

Revision. The four velocity-source and horizon-rule rows of Table 3 (main text) and Table 7 (appendix) are replaced by the unified results with the tolerance and horizon rule stated, and the discussion in Section 4.6 is updated; Table 5 adds the latency of the restricted-vocabulary parallel heads.

Response to Reviewer e7mr (2/3)

[Q1] Wall-clock comparison with batched FlowCast, distinguishing faithful implementations from adaptations.

[A2] Thank you for the suggestion. We implemented the batched verification of FlowCast following the paper, measured its quality on the evaluation images of Table 1 and its latency against the reference on the same GPU, and the revision states for every baseline whether it follows the original implementation or is adapted to this backbone.

(1) FlowCast with batched verification as published. This version follows Algorithm 1 of FlowCast. It drafts all remaining states, verifies them in batched backbone calls, accepts the prefix under the velocity MSE threshold (including 0.07, the value used for BAGEL editing in the FlowCast paper), and corrects one step with the verified velocity after a rejection. At batch size one, the batched call reproduces the per-state call element-wise. Speedups are wall-clock ratios to the reference.

SettingNYUv2 depth ↓NYUv2 normals ↓Calls (depth / normals) ↓Speedup (depth / normals) ↑
Reference (49 steps)4.0513.88981.0×
FlowCast, serial implementation in Table 13.9213.85100 / 1021.0×
FlowCast, batched as published, ε=0.023.88–413 / –0.27× / –
FlowCast, batched as published, ε=0.044.1113.96320 / 3950.34× / 0.28×
FlowCast, batched as published, ε=0.074.3614.32280 / 3390.39× / 0.33×
FlowCast, batched as published, ε=0.124.6314.87251 / 2960.43× / 0.37×
PercepDraft (Table 1)3.8713.7713.4 / 15.86.3× / 5.4×
Panoptic (COCO)PQ ↑Calls ↓Speedup ↑
Reference (49 steps)51.4981.0×
FlowCast, serial implementation in Table 148.1107.71.0×
FlowCast, batched as published, ε=0.0251.710210.13×
FlowCast, batched as published, ε=0.0751.17230.18×
FlowCast, batched as published, ε=0.1251.16360.20×
PercepDraft (Table 1)51.338.72.0×

Batched FlowCast is slower than the reference at every threshold, at 0.27–0.43× on geometry and 0.13–0.20× on panoptic segmentation, and at ε=0.07 it is also slower on KITTI and ScanNet (0.38× and 0.40×). It evaluates 2.6 to 10.4 times as many states per image as the reference, and a single SenseNova-Vision backbone call already runs near the arithmetic peak of the GPU, so packing K states into one call costs close to K separate calls; batching reduces the number of sequential calls but not the total computation. The serial implementation in Table 1 is therefore the more favorable of the two for FlowCast. This also shows that, on a backbone whose single call saturates the device, acceleration has to come from evaluating fewer states. PercepDraft verifies each segment once at its landing point and sets the segment length from the predicted errors, which is where its speedup comes from.

(2) Implementation of each baseline. Appendix C.2.1 of the revision states the implementation of each baseline as follows.

MethodImplementation
FlowCastTable 1 uses a serial implementation (drafted states verified one at a time, stopping at the first failure, relative ℓ2 residual on the velocity); this response adds the batched verification as published (all remaining states verified in a batch, velocity MSE)
DeepCache, TeaCache, TaylorSeerReimplemented on the MoT backbone of SenseNova-Vision with their published caching and reuse rules (the original implementations target U-Net or DiT backbones)
FastV, AgilePrunerRun with their official configurations
MedusaTrained with the same data, number of steps and number of heads as PercepDraft's token drafter, predicting over the full vocabulary

We thank the reviewer again for the suggestion; this comparison makes our evaluation more complete.

Revision. We add Table 16 (Appendix D) with the quality and latency of batched FlowCast, and Appendix C.2.1 states the implementation of each baseline.

Response to Reviewer e7mr (3/3)

[Q2] Velocity-source and horizon combinations at identical tolerances, the token branch versus restricted-vocabulary parallel heads, and the speedup remaining over tuned reduced-step schedules including the eight-step Marigold baseline.

[A3] Thank you for the questions, which we answer in turn.

(1) Velocity source and horizon rule at identical tolerances. The table in W1 (1) reports the quality, calls and latency of the four combinations at the same per-task tolerance and with the same training setting. PercepDraft (trajectory drafter with trusted horizon) reaches the best or near-best quality with the fewest calls on all three tasks. Against the simplest alternative, constant velocity with a fixed horizon, it uses one third fewer calls on depth, raising the speedup from 4.5× to 6.3×, and about half and 70% of the calls on normals and panoptic segmentation. The trusted horizon provides most of the reduction in calls, and the trajectory drafter's velocities further improve quality.

(2) Wall-clock gain of the token branch over restricted-vocabulary parallel heads. With the same verifier and runtime, PercepDraft's token branch takes 1.25 / 6.65 / 23.5 s per image on LVIS / VisDrone / Dense200, the restricted-vocabulary parallel heads 1.29 / 6.82 / 23.4 s, and the full-vocabulary Medusa heads 1.50 / 8.33 / 28.7 s. The restricted vocabulary and the chained heads are both part of PercepDraft's token drafter and together are 1.20–1.25× faster than the existing Medusa heads. On top of the restricted vocabulary, chaining raises the committed tokens per call from 3.46 to 3.54, giving a further 3% and 2.5% on LVIS and VisDrone, and is on par on Dense200.

(3) Remaining speedup over tuned reduced-step schedules. On the evaluation images of Table 1, we compare CFG at every step, CFG only for t>0.9, and no CFG between 6 and 49 steps, and also apply PercepDraft to the unguided samplers without a trajectory drafter, using constant-velocity drafts and landing-point verification only. CFG denotes classifier-free guidance, which the released SenseNova-Vision sampler applies at every step. Calls are backbone evaluations per image (two per step with CFG), and speedups are wall-clock ratios to the reference on the same GPU.

SettingNYUv2 depth ↓KITTI ↓NYUv2 normals ↓ScanNet ↓Calls (depth / normals) ↓Speedup (depth / normals) ↑
Reference (CFG, 49 steps)4.056.2313.8812.69981.0×
CFG, 24 steps4.167.6215.6313.68482.0×
CFG, 12 steps6.4520.1622.9421.07243.7× / 3.8×
CFG for t>0.9, 49 steps4.066.3013.8512.67651.5× / 1.4×
No CFG, 12 steps4.095.9913.9312.67126.5× / 6.7×
No CFG, 6 steps3.945.8313.7912.55610.7× / 11.0×
PercepDraft (Table 1)3.875.6013.7712.6213.4 / 15.86.3× / 5.4×
No CFG, 12 steps + PercepDraft3.855.5913.6112.39414.1× / 14.5×
No CFG, 6 steps + PercepDraft3.755.5913.6312.38316.5× / 16.8×

On panoptic segmentation, the reference (CFG, 49 steps) reaches 51.4 PQ and PercepDraft 51.3 PQ at 2.0×. The reduced-step schedules give the following PQ (speedup).

Panoptic PQ ↑ (speedup ↑)20 steps16 steps8 steps
CFG48.1 (1.8×)46.0 (2.1×)36.9 (2.8×)
CFG for t>0.947.8 (2.2×)46.5 (2.5×)37.8 (3.1×)
No CFG49.6 (2.5×)49.6 (2.8×)47.4 (3.4×)
Marigold v1.1 normalsNYUv2 ↓ScanNet ↓Calls ↓Speedup ↑
Reference (50 steps)16.9815.37501.0×
Uniform 8 steps16.5315.0784.6×
PercepDraft16.5415.045.15.8×
Marigold v1.0 depthNYUv2 ↓KITTI ↓Calls ↓Speedup ↑
Reference (50-step DDIM)6.0212.97501.0×
10-step DDIM6.15–104.0×
PercepDraft5.9112.905.45.6×

With CFG at every step, as released, the fastest reduced-step schedule that matches the reference is CFG for t>0.9 over 49 steps at 1.5×/1.4×, while PercepDraft reaches 6.3×/5.4× with lower error. On panoptic segmentation no reduced-step schedule reaches the reference quality. Without CFG few steps work on geometry but lose at least 1.8 PQ on panoptic segmentation; PercepDraft is orthogonal and reduces the 12- and 6-step unguided samplers to 4 and 3 calls with lower error (up to 16.8×). On Marigold, PercepDraft is 1.26× faster than the tuned 8-step schedule at comparable quality for v1.1 normals, and both faster and more accurate than 10-step DDIM for v1.0 depth. In neither case does PercepDraft require choosing a step count for the model or the task.

Revision. As in W1; we add Table 15 (Appendix D) with the reduced-step schedules, and the Marigold comparison is in Appendix B.1.

Reviewer_ubuyOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PercepDraft accelerates generative perception using two lightweight drafters while keeping the backbone frozen. For structured detection outputs, a token drafter proposes future tokens and the backbone verifies a matching prefix. For dense outputs, a trajectory drafter proposes future denoising velocities, selects a horizon using predicted errors, and verifies the landing point. The paper evaluates detection, panoptic segmentation, depth, and surface normals across several backbones, reporting reduced end-to-end latency with generally limited changes in task metrics.

Critical strengths
  1. The panoptic segmentation result shows a useful quality–efficiency trade-off: on SenseNova-Vision, latency decreases from 12.1 to 5.91 seconds while PQ changes from 51.4 to 51.3.
  2. The paper evaluates multiple tasks and backbones. Its horizon and verification ablations provide evidence that the dense branch can reduce backbone calls without simply accepting every drafted trajectory.
Critical weaknesses
  1. The additional value over few-step inference is not sufficiently established for depth and normals. The largest speedups are measured against 49/50-step reference runs. Yet the paper itself reports that a fixed eight-step schedule for Marigold normal estimation matches its 50-step reference quality at 4.6× speedup, compared with 5.8× for PercepDraft. Figure 7 also shows that early stopping is competitive on geometry tasks. Although the paper includes these analyses, it does not clearly establish a consistent advantage for drafting and verification over tuned few-step inference at matched accuracy and end-to-end latency across the full depth and normal benchmarks.
  2. The detection branch does not always reproduce the original greedy output in the measured implementation. The authors report finite-precision differences when verification evaluates multiple query rows together. On SenseNova-Vision’s Dense200 benchmark, F1 decreases from 81.3 to 79.4. This behavior is disclosed and investigated, but it qualifies the practical output-preservation claim.
  3. The depth-related literature discussion could be more complete. Pixel-Perfect Depth and InfiniDepth are neither cited nor discussed. The authors should clarify how these works relate to the paper’s depth setting and compare them where their evaluation and inference protocols are compatible.
Questions for authors
  1. On the full depth and normal benchmarks, how does PercepDraft compare with tuned fixed-step and early-stopping schedules at matched task quality and measured end-to-end latency?
  2. How do Pixel-Perfect Depth and InfiniDepth relate to the proposed depth setting? If a direct comparison is unsuitable, what differences in task definition or evaluation protocol prevent it?
  3. For PercepDraft itself, how do detection quality and latency change if the described near-tie correction is enabled?
7 comments

Response to Reviewer ubuy (1/3)

We sincerely thank the reviewer for the constructive comments and for recognizing the quality–efficiency trade-off on panoptic segmentation, the evaluation across multiple tasks and backbones, and the horizon and verification ablations. All changes are marked in blue in the revision.

[W1&Q1] Value over tuned reduced-step schedules and early stopping on depth and normals.

[A1] The 49-step schedule is the one released with SenseNova-Vision, which is why we use it as the reference. We agree that tuned reduced-step inference and early stopping should also be compared, and added both under the evaluation setting of Table 1 (NYUv2 and KITTI depth, NYUv2 and ScanNet normals).

(1) Reduced-step schedules. CFG denotes classifier-free guidance, which the released SenseNova-Vision sampler applies at every step; "CFG for t>0.9" applies it only for t>0.9. Calls are backbone evaluations per image (two per step with CFG), and speedups are wall-clock ratios to the reference on the same GPU. Depth is AbsRel and normals are mean angular error.

SettingNYUv2 depth ↓KITTI ↓NYUv2 normals ↓ScanNet ↓Calls (depth / normals) ↓Speedup (depth / normals) ↑
Reference (CFG, 49 steps)4.056.2313.8812.69981.0×
CFG, 24 steps4.167.6215.6313.68482.0×
CFG, 12 steps6.4520.1622.9421.07243.7× / 3.8×
CFG for t>0.9, 49 steps4.066.3013.8512.67651.5× / 1.4×
No CFG, 49 steps4.246.1714.0812.80491.9×
No CFG, 12 steps4.095.9913.9312.67126.5× / 6.7×
No CFG, 6 steps3.945.8313.7912.55610.7× / 11.0×
PercepDraft (Table 1)3.875.6013.7712.6213.4 / 15.86.3× / 5.4×
No CFG, 49 steps + PercepDraft4.035.7513.8512.5810.7 / 17.77.4× / 5.0×
No CFG, 12 steps + PercepDraft3.855.5913.6112.39414.1× / 14.5×
No CFG, 6 steps + PercepDraft3.755.5913.6312.38316.5× / 16.8×

Schedules with CFG degrade quickly as steps are removed: KITTI AbsRel rises from 6.23 to 7.62 at 24 steps and exceeds 20 at 12 steps. CFG for t>0.9 over 49 steps keeps the reference quality but gives only 1.4–1.5×. Without CFG the geometry trajectories are nearly straight and few-step schedules are effective on geometry, but the same setting is at least 1.8 PQ below the reference on panoptic segmentation, so it is a configuration to be chosen per task. PercepDraft has lower error than the reference on all four benchmarks and is orthogonal to the sampler configuration. Without a trajectory drafter, using constant-velocity drafts and landing-point verification only, it reduces the 49-step unguided sampler from 49 to 11–18 calls with lower error on all four benchmarks (NYUv2 depth 4.24 to 4.03, KITTI 6.17 to 5.75, NYUv2 normals 14.08 to 13.85, ScanNet 12.80 to 12.58), and reduces the 12- and 6-step unguided samplers to 4 and 3 calls, again with lower error (for 6 steps, NYUv2 depth 3.94 to 3.75, KITTI 5.83 to 5.59, NYUv2 normals 13.79 to 13.63, ScanNet 12.55 to 12.38), reaching 16.5× and 16.8×.

(2) Early stopping. Early stopping outputs the clean-output estimate x − t·v at step k instead of completing the schedule.

SettingNYUv2 depth ↓KITTI ↓NYUv2 normals ↓ScanNet ↓Panoptic PQ ↑Speedup (depth / normals) ↑
Reference (49 steps)4.056.2313.8812.6951.41.0×
Early stop at step 53.795.4814.4212.6831.87.9× / 8.2×
Early stop at step 83.745.4313.7512.3839.25.4× / 5.6×
Early stop at step 404.026.1313.7712.5950.3≈1.2×
PercepDraft (Table 1)3.875.6013.7712.6251.36.3× / 5.4×
No CFG, 6 steps + PercepDraft3.755.5913.6312.38–16.5× / 16.8×

Across tasks, PercepDraft is ahead of early stopping for two reasons. Imbalance across tasks. Step 8 is the best for geometry, but it lowers panoptic PQ from 51.4 to 39.2, and panoptic segmentation needs step 40 (80 calls) to reach 50.3 with almost no speedup; even within geometry, step 5 already beats the reference on depth (3.79) but not on normals (14.42 vs. 13.88). No single stopping step is therefore robust across tasks. No adaptation to the image. A fixed stopping step is the same for every image, whereas the best cutoff varies by image (Section 4.5); PercepDraft sets the length of each segment from the predicted error on that image and checks it at the landing point. Even against the best geometry stopping step chosen in hindsight, the 6-step unguided sampler with PercepDraft reaches comparable errors with 3 calls and runs 3× faster (16.5× vs. 5.4×).

Revision. We add Table 15 (Appendix D) with reduced-step schedules and early stopping and cite it in Section 4.5.

Response to Reviewer ubuy (2/3)

[W2&Q3] Detection quality and latency of PercepDraft with the near-tie guard enabled.

[A2] Thank you for acknowledging our disclosure and analysis of this effect. We explain where the difference comes from and add the results of PercepDraft itself with the near-tie guard.

(1) Source of the difference. The difference comes from batched verification itself, not from the accepted drafts. Feeding the greedy output back through the multi-query verification forward (teacher forcing) changes the argmax at 1.0% of positions in bf16; every change occurs where the top-two logits are within bf16 resolution, almost always on a coordinate digit, and moves a box edge by one pixel. Qwen2.5-VL and Rex-Omni show the same pattern. Under exact arithmetic, the acceptance rule is token-for-token equivalent to greedy decoding (the induction argument in Section 3.4).

(2) Near-tie guard. The guard recomputes positions whose top-two logit margin falls below a threshold with an exact single-query step, so it triggers only at the rare near-tied positions. These positions are determined by batched verification and do not depend on the draft source. With prompt-lookup drafts, enabling the guard restores Dense200 F1 from 79.3 to 80.4 at about 15% of the throughput gain. The results for PercepDraft itself are below.

SettingLVIS ↑VisDrone ↑Dense200 ↑s/img ↓ (LVIS / VisDrone / Dense200)
Greedy decoding (reference)81.866.081.31.91 / 10.7 / 30.4
PercepDraft82.365.679.41.25 / 6.65 / 23.5
PercepDraft + near-tie guard82.365.880.41.32 / 7.05 / 24.3

With the guard, PercepDraft recovers 80.4 F1 on Dense200 and 65.8 on VisDrone and keeps 82.3 on LVIS. Because the guard triggers at very few positions, the latency rises only slightly, and PercepDraft remains 1.45× / 1.52× / 1.25× faster than greedy decoding. When the greedy output must be reproduced exactly, the guard recovers most of the finite-precision difference at a small latency cost.

(3) Wording. Section 1 and Section 3.4 already qualify the claim as "apart from rare finite-precision changes at near-tied logits", and Appendix C.2.3 gives the analysis above together with the guard.

Revision. Appendix C.2.3 adds the quality and latency of PercepDraft with the near-tie guard.

Response to Reviewer ubuy (3/3)

[W3&Q2] Relation to Pixel-Perfect Depth and InfiniDepth.

[A3] Thank you for pointing us to these two works. Both are strong and timely contributions to monocular depth estimation. Pixel-Perfect Depth shows that flow matching directly in pixel space, guided by semantic features, yields sharp depth without the flying pixels of latent-space generation, and InfiniDepth's implicit decoder makes depth queryable at arbitrary resolution. The revision discusses and cites both in Related Work and Table 2, and we also evaluated them with their official code and weights.

(1) Relation to our setting. Pixel-Perfect Depth (Xu et al., NeurIPS 2025) generates depth with flow matching in pixel space and prompts the DiT with features of a vision foundation model. InfiniDepth (Yu et al., CVPR 2026) decodes a neural implicit field from DINOv3 features and predicts depth at arbitrary resolution in a single forward pass. Both are specialized depth models, whereas our setting accelerates the inference of an existing multi-task generative perception model without changing its weights and while preserving its outputs.

(2) Accuracy. Both follow an evaluation protocol compatible with ours (zero-shot, affine-invariant depth), so we ran their official models under the evaluation setting of Table 1 with the alignment used in each paper. Pixel-Perfect Depth predicts affine-invariant log depth and is aligned with RANSAC in log space; the relative-depth model of InfiniDepth predicts disparity and is aligned by least squares in disparity, as for SenseNova-Vision.

ModelNYUv2 AbsRel ↓ / δ1 ↑KITTI AbsRel ↓ / δ1 ↑
Pixel-Perfect Depth4.26 / 97.37.57 / 94.3
InfiniDepth4.44 / 97.47.16 / 93.1
SenseNova-Vision (reference)4.05 / 97.96.23 / 95.0
SenseNova-Vision + PercepDraft3.87 / 98.15.60 / 96.0

Under the same evaluation on these images, SenseNova-Vision with PercepDraft obtains lower AbsRel and higher δ1. As the two are dedicated depth models with a different focus (sharp geometry and arbitrary resolution), this mainly shows that the accelerated generalist stays highly competitive with dedicated depth models. Following the convention of Table 2, we also add the numbers reported in their papers to Table 2.

(3) PercepDraft on Pixel-Perfect Depth. Pixel-Perfect Depth integrates a flow-matching trajectory with Euler steps, so the dense-trajectory branch of PercepDraft applies to it directly. We apply it to the Pixel-Perfect Depth sampler without a trajectory drafter, using constant-velocity drafts and landing-point verification only.

Pixel-Perfect Depth samplerNYUv2 AbsRel ↓ / δ1 ↑KITTI AbsRel ↓ / δ1 ↑Calls ↓Speedup ↑
8 steps4.28 / 97.37.68 / 94.181.0×
8 steps + PercepDraft4.24 / 97.47.51 / 94.441.75×
16 steps4.33 / 97.37.77 / 93.9161.0×
16 steps + PercepDraft4.23 / 97.47.55 / 94.352.7×

PercepDraft reduces the calls of Pixel-Perfect Depth to one half to one third, a 1.75–2.7× speedup, with slightly better accuracy and no rejected draft, which indicates that its pixel-space trajectories are also nearly straight. Compared with the released 4-step configuration of Pixel-Perfect Depth, PercepDraft reaches better accuracy with a similar 4 to 5 calls (NYUv2 AbsRel 4.24 and 4.23 vs. 4.26, KITTI 7.51 and 7.55 vs. 7.57). The dense-trajectory branch of PercepDraft thus transfers to another generative depth model without retraining. InfiniDepth predicts depth in one forward pass and has no iterative trajectory, so it lies outside the scope of PercepDraft.

We thank the reviewer again for these references; the experiment also illustrates the generality of PercepDraft.

Revision. We discuss both works in Related Work, add their reported numbers to Table 2, and add the Pixel-Perfect Depth acceleration results to Appendix B.1.

Reviewer_ZxmmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The work proposes PercepDraft to accelerate generative perception with a frozen backbone. A token drafter predicts multiple tokens for structured detection outputs, while a trajectory drafter predicts future denoising velocities and estimates their errors to select a trusted horizon. The backbone verifies token prefixes or the landing point of a dense trajectory. The exps cover detection, panoptic segmentation, depth, and surface normals across several backbones. and rhe reported results show substantial reductions in inference time relative to the original decoding procedures, with generally limited changes in task performance.

Critical strengths

S1: The reported acceleration is practically meaningful and extends across several tasks and backbones. In Table 1, SenseNova-Vision depth latency decreases from 8.66 to 1.38 seconds, while panoptic latency decreases from 12.1 to 5.91 seconds with PQ changing from 51.4 to 51.3. Training lightweight drafters on the frozen backbone's outputs without task labels makes the approach attractive for adapting existing systems.

S2: The paper includes useful diagnostics of when drafting works and when verification matters. For example, Table 3 shows that force-accepting 16-step drafts reduces panoptic PQ to 28.7. The discussion also explicitly acknowledges that landing-point verification does not certify intermediate states. These analyses help explain the different behavior of geometry and segmentation outputs.

Critical weaknesses

W1: Some baseline settings make the size of the comparative advantage difficult to assess. Appendix C.2.5 states that the FlowCast timing in Table 1 uses a serial implementation, although the original method verifies drafted states in parallel. Equivalent backbone-call counts alone cannot establish the latency of that parallel implementation. The appendix also describes FlowCast's acceptance criterion as MSE on states, whereas Algorithm 1 and Equation 5 of FlowCast use MSE between velocities. It is unclear whether this is a wording error or an implementation difference. The reduced-step controls also leave an important gap. Appendix B.1 reports that an eight-step Marigold normals schedule matches the 50-step reference at a 4.6× speedup, compared with 5.8× for PercepDraft. This corresponds to an approximately 1.26× incremental speedup over the reduced-step control. Figure 7 includes early-stopping results, and Table 6 evaluates a half-length schedule for panoptic segmentation, but neither provides a matched quality–latency comparison across the main SenseNova-Vision dense tasks. This matters because the paper's main contribution is improved inference efficiency.

W2: The recovery cost after a rejected dense draft is not fully accounted for. Section 3.3 states that rejection incurs no additional backbone call because the sampler reuses the cached anchor velocity for one exact fallback step. While this is sufficient to compute the fallback state, it is unclear how the velocity at the new anchor is obtained before drafting the next segment. The velocity evaluated at the rejected draft's landing point generally does not provide the velocity at this fallback state. Please clarify the complete rejection-and-resumption procedure and explain how all associated backbone evaluations are counted in the reported call statistics and the expected-progress analysis.

Questions for authors

Q1: Which residual was used in the FlowCast implementation, state MSE or velocity MSE? Could the authors report its latency with batched verification under the same hardware and quality target, or explicitly limit the comparison to the serial adaptation?

Q2: At comparable output quality, how much latency advantage remains over shorter schedules that still reach the terminal time? Please compare these controls and PercepDraft on the same samples, especially for SenseNova-Vision geometry and panoptic segmentation.

Q3: After rejecting a draft longer than one step, how is the backbone velocity at the fallback state obtained? Please provide pseudocode and clarify whether any additional evaluation is included in both the reported call counts and the analysis in Section 3.3.

7 comments

Response to Reviewer Zxmm (1/3)

We sincerely thank the reviewer for the thorough review and for recognizing the practical acceleration across tasks and backbones, the label-free training of lightweight drafters on a frozen backbone, and our analysis of when drafting works and when verification matters. All changes are marked in blue in the revision.

[W1&Q1] Residual used for FlowCast, and its latency with batched verification.

[A1] Thank you for checking this against the FlowCast paper. The sentence in Appendix C.2.5 describes the threshold of the original FlowCast, and "MSE on the state" is a wording error: FlowCast thresholds the MSE between velocities. Our implementation follows the original and also compares velocities; it only replaces the MSE with a relative ℓ2 distance, as the second half of the same sentence states. The revision corrects the wording. For the serial versus parallel question, we additionally implemented FlowCast with batched verification as published, measured its quality on the evaluation images of Table 1, and measured its latency against the reference on the same GPU.

(1) Residual. Our implementation keeps the anchor velocity fixed to draft the subsequent states, evaluates the backbone velocity at each drafted state, compares it with this fixed velocity, and accepts the longest passing prefix, as in Algorithm 1 of FlowCast. The batched version below uses the velocity MSE and reaches the same conclusion, so the choice of distance does not affect the comparison. The threshold sweep in the appendix covers 0.15 to 0.5, and the serial implementation needs 100 to 120 calls at every threshold, never fewer than the 98 calls of the unaccelerated model.

(2) The serial implementation favors FlowCast in call count. It verifies drafted states one at a time and stops at the first failure, whereas the published procedure evaluates all drafted states at once and can only use more calls. The serial implementation therefore only forgoes FlowCast's parallelism and never adds computation to it.

(3) Batched verification as published. We implemented the batched verification of FlowCast following the paper. It drafts all remaining states, verifies them in batched backbone calls, accepts the prefix under the velocity MSE threshold (including 0.07, the value used for BAGEL editing in the FlowCast paper), and corrects one step with the verified velocity after a rejection. At batch size one, the batched call reproduces the per-state call element-wise. Results on the evaluation images of Table 1 are below; speedups are wall-clock ratios to the reference.

SettingNYUv2 depth ↓NYUv2 normals ↓Calls (depth / normals) ↓Speedup (depth / normals) ↑
Reference (49 steps)4.0513.88981.0×
FlowCast, serial implementation in Table 13.9213.85100 / 1021.0×
FlowCast, batched as published, ε=0.023.88–413 / –0.27× / –
FlowCast, batched as published, ε=0.044.1113.96320 / 3950.34× / 0.28×
FlowCast, batched as published, ε=0.074.3614.32280 / 3390.39× / 0.33×
FlowCast, batched as published, ε=0.124.6314.87251 / 2960.43× / 0.37×
PercepDraft (Table 1)3.8713.7713.4 / 15.86.3× / 5.4×
Panoptic (COCO)PQ ↑Calls ↓Speedup ↑
Reference (49 steps)51.4981.0×
FlowCast, serial implementation in Table 148.1107.71.0×
FlowCast, batched as published, ε=0.0251.710210.13×
FlowCast, batched as published, ε=0.0751.17230.18×
FlowCast, batched as published, ε=0.1251.16360.20×
PercepDraft (Table 1)51.338.72.0×

Batched FlowCast is slower than the reference at every threshold, at 0.27–0.43× on geometry and 0.13–0.20× on panoptic segmentation; at ε=0.07 it is also slower on KITTI and ScanNet (0.38× and 0.40×). It evaluates 2.6 to 10.4 times as many states per image as the reference, and a single SenseNova-Vision backbone call already runs near the arithmetic peak of the GPU, so packing K states into one call costs close to K separate calls. Batching reduces the number of sequential calls but not the total computation, so the serial implementation in Table 1 is the more favorable of the two for FlowCast. This also shows that, on a backbone whose single call saturates the device, acceleration has to come from evaluating fewer states rather than from evaluating them in parallel. PercepDraft verifies each segment once at its landing point and sets the segment length from the predicted errors, which is where its speedup comes from, so the conclusion of Table 1 also holds against FlowCast as published. We thank the reviewer for suggesting this comparison, which makes our evaluation more complete.

Revision. We correct the description in Appendix C.2.5 and add Table 16 (Appendix D) with the quality and latency of batched FlowCast.

Response to Reviewer Zxmm (2/3)

[W1&Q2] Remaining speedup over reduced-step schedules and early stopping at comparable quality.

**[A2]**We added reduced-step schedules that reach the terminal time, early stopping, and the Marigold control on the Table 1 images.

(1) Reduced-step schedules. We compare classifier-free guidance (CFG) at every step as released, CFG only for t>0.9, and no CFG, and apply PercepDraft to the unguided samplers without a trajectory drafter (constant-velocity drafts and landing-point verification only). Calls count backbone evaluations (two per step with CFG), and speedups are relative to the reference on the same GPU; depth is AbsRel and normals are mean angular error.

SettingNYUv2 depth ↓KITTI ↓NYUv2 normals ↓ScanNet ↓Calls (depth / normals) ↓Speedup (depth / normals) ↑
Reference (CFG, 49 steps)4.056.2313.8812.69981.0×
CFG, 24 steps4.167.6215.6313.68482.0×
CFG for t>0.9, 49 steps4.066.3013.8512.67651.5× / 1.4×
No CFG, 12 steps4.095.9913.9312.67126.5× / 6.7×
No CFG, 6 steps3.945.8313.7912.55610.7× / 11.0×
PercepDraft (Table 1)3.875.6013.7712.6213.4 / 15.86.3× / 5.4×
No CFG, 12 steps + PercepDraft3.855.5913.6112.39414.1× / 14.5×
No CFG, 6 steps + PercepDraft3.755.5913.6312.38316.5× / 16.8×

Panoptic PQ (speedup); reference 51.4, PercepDraft 51.3 at 2.0×.

Panoptic PQ ↑ (speedup ↑)20 steps16 steps8 steps
CFG48.1 (1.8×)46.0 (2.1×)36.9 (2.8×)
CFG for t>0.947.8 (2.2×)46.5 (2.5×)37.8 (3.1×)
No CFG49.6 (2.5×)49.6 (2.8×)47.4 (3.4×)

On panoptic segmentation every reduced-step schedule stays at least 1.8 PQ below the reference, and at a similar 2× speedup, CFG with 16 steps loses 5.3 PQ while PercepDraft keeps 51.3 PQ. On geometry, schedules with CFG degrade quickly, and only CFG for t>0.9 over 49 steps keeps the reference quality, at 1.4–1.5×. Without CFG few steps work well on geometry but not on panoptic segmentation, so this is a per-task configuration rather than a general accelerator. PercepDraft is orthogonal to it. Without a trajectory drafter, it reduces the 12- and 6-step unguided samplers to 4 and 3 calls with lower error on all four benchmarks, reaching 16.5× and 16.8×.

(2) Early stopping. Early stopping outputs the clean-output estimate x − t·v at step k; the panoptic results follow Table 11, with step 5 added.

SettingNYUv2 depth ↓KITTI ↓NYUv2 normals ↓ScanNet ↓Panoptic PQ ↑Speedup (depth / normals) ↑
Reference (49 steps)4.056.2313.8812.6951.41.0×
Early stop at step 53.795.4814.4212.6831.87.9× / 8.2×
Early stop at step 83.745.4313.7512.3839.25.4× / 5.6×
Early stop at step 404.026.1313.7712.5950.3≈1.2×
PercepDraft (Table 1)3.875.6013.7712.6251.36.3× / 5.4×
No CFG, 6 steps + PercepDraft3.755.5913.6312.38–16.5× / 16.8×

Across tasks, PercepDraft is ahead of early stopping for two reasons. Imbalance across tasks. Step 8 is the best for geometry, but it lowers panoptic PQ from 51.4 to 39.2, and panoptic segmentation needs step 40 (80 calls) to reach 50.3 with almost no speedup; even within geometry, step 5 already beats the reference on depth (3.79) but not on normals (14.42 vs. 13.88). No single stopping step is therefore robust across tasks. No adaptation to the image. A fixed stopping step is the same for every image, whereas the best cutoff varies by image (Section 4.5); PercepDraft sets the length of each segment from the predicted error on that image and checks it at the landing point. Even against the best geometry stopping step chosen in hindsight, the 6-step unguided sampler with PercepDraft reaches comparable errors with 3 calls and runs 3× faster (16.5× vs. 5.4×).

(3) Marigold.

Marigold v1.1 normalsNYUv2 ↓ScanNet ↓Calls ↓Speedup ↑
Reference (50 steps)16.9815.37501.0×
Uniform 8 steps16.5315.0784.6×
PercepDraft16.5415.045.15.8×
Marigold v1.0 depthNYUv2 ↓KITTI ↓Calls ↓Speedup ↑
Reference (50-step DDIM)6.0212.97501.0×
10-step DDIM6.15–104.0×
PercepDraft5.9112.905.45.6×

Even on Marigold v1.1 normals, trained for few-step inference, PercepDraft matches the tuned 8-step quality with 5.1 calls (1.26× faster); on v1.0 depth, the tuned 10-step DDIM is slower and less accurate.

Revision. We add Table 15 (Appendix D) with reduced-step schedules and early stopping and cite it in Section 4.5.

Response to Reviewer Zxmm (3/3)

[W2&Q3] How the velocity at the fallback state is obtained after a rejection, and whether this call is counted.

[A3] Thank you for the careful reading. The velocity at the fallback state comes from one regular backbone call, and this call is included in every reported call count and latency. The call counts in all tables are measured by counting every backbone evaluation, including discarded verifications and this call, so the reported numbers do not change. The text of Section 3.3 described only the fallback step, and its expected-progress formula omitted this call; the revision corrects both.

(1) Full procedure.

x <- x_0; i <- 0; plain <- false
v <- F(x, t_0)                                  # call
while i < N:
    H <- 1
    if 2 <= i <= N-3 and not plain:
        (dv_1..K, e_1..K) <- g(x, v, v_prev, t_i)
        H <- min(trusted_horizon(e, rho*tau), N-1-i)
    if H = 1:                                   # exact step
        x <- Psi_i(x, v); i <- i+1; plain <- false
        if i < N: v <- F(x, t_i)                # call
        continue
    x_hat <- roll out steps i..i+H-1 from x with v, v+dv_1, ..., v+dv_{H-1}
    v* <- F(x_hat, t_{i+H})                     # call (verification)
    if ||v* - (v+dv_H)|| <= tau ||v*||:         # accept
        x <- x_hat; i <- i+H; v <- v*           # the verifying call is the next anchor's call
    else:                                       # reject, v* is discarded
        x <- Psi_i(x, v); i <- i+1              # fallback with the cached anchor velocity, no call
        v <- F(x, t_i)                          # call at the fallback state
        plain <- true                           # one exact step before drafting resumes

(2) Cost. A rejection spends three calls to advance two steps, namely the discarded verification, the call at the fallback state, and the call after the following exact step, which is one call more than sequential sampling. With acceptance probability q and horizon H, the expected progress per call is (2+q(H−2))/(3−2q); it equals H at q=1 and exceeds one exactly when qH>1. The main panoptic configuration has 2.3 rejections per image on average, which account for 4.6 of its 38.7 calls (with CFG, each step has two branches and counts as two calls).

To validate this cost model, we counted the calls per image of training-free constant-velocity drafting on the three tasks. It uses a fixed horizon and exactly the same landing-point verification and fallback as PercepDraft. Simulating the procedure above with the measured acceptance rate q predicts the measured calls within 5%, whereas omitting the call at the fallback state undercounts by 0.7 to 13.4 calls, increasingly so as rejections become more frequent. Across all 16 runs (3000 images), the measured backbone forwards equal exactly steps − steps saved by accepted segments + rejections (calls are twice this with CFG).

Setting (constant-velocity drafts)Acceptance rate qRejections / imageMeasured callsPredicted, corrected modelPredicted without the fallback call
Depth, fixed 8 steps, τ=0.40.950.3420.020.019.3
Depth, fixed 8 steps, τ=0.30.732.1130.329.626.0
Normals, fixed 8 steps, τ=0.40.752.0230.028.826.0
Normals, fixed 8 steps, τ=0.30.633.0034.035.728.0
Panoptic, fixed 6 steps, τ=0.30.752.5135.735.030.7
Panoptic, fixed 8 steps, τ=0.20.396.7355.054.941.6

Revision. We rewrite the procedure and the expected-progress formula in Section 3.3 and state that all call counts are measured; the fallback descriptions in Section 3.5 and Appendix B.2 are updated accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.