acceptodds
Under review as a conference paper at ICLR 2027

Where Does Occupancy Latency Go? A Per-Stage Decomposition of 3D Occupancy Inference

Abstract

Evaluation of 3D semantic occupancy prediction relies almost exclusively on accuracy metrics, namely mean intersection over union (mIoU) and its ray-based variants, measured on server-grade GPUs. Deployment-related metrics do appear in the literature, but they are collected on an ad hoc basis across different hardware platforms under unspecified settings. Among the works we surveyed, none reports these metrics as a function of computational budget. To address this gap, we provide the measurements needed for budget-aware evaluation by decomposing occupancy inference latency stage by stage. We instrument six public configurations and attribute latency to six concrete stages, repeat runs under a common fixed protocol, report median and 95th-percentile (p95) latency, and account for the residual time outside the measured stages. Extensive experiments show that the 2D image backbone, rather than the occupancy head, dominates single-frame latency. A two-frame temporal window shifts the dominant cost to view transformation, while scaling up the backbone makes image feature extraction a co-dominant cost again. Comparisons among variants within the same architecture lineage further show that the differences are concentrated primarily in the bird's-eye-view (BEV) encoder backbone and occupancy head, where they correspond to approximately constant absolute latency savings. We use this decomposition to introduce a new reporting convention that expresses accuracy as an explicit function of deployment budget. We apply it to Occ3D-nuScenes and make the measurement framework and results publicly available.

Primary area:
datasets and benchmarks
Keywords:
3D Semantic Occupancy Prediction, Inference Latency, Stage-level Profiling, Budget-aware Evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_jdkWOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper profiles six BEVDet-OCC and FlashOCC configurations on Occ3D-nuScenes. It decomposes inference latency into six stages, reports residual time and latency variability, and selects the most accurate measured configuration under specified median-latency budgets.

Critical strengths

The measurements use a common hardware and timing protocol. Reporting the residual and additional runs makes the limits of the stage-level comparisons more visible.

Critical weaknesses

Limited insight beyond profiling. The paper just reports which stages are slower in existing models, without showing what can be learned beyond the timing differences themselves. For example, it identifies the image backbone as a bottleneck in the single-frame setting but does not examine whether reducing its cost would improve end-to-end latency or affect accuracy.

Narrow scope. Only BEVDet-OCC and FlashOCC are evaluated. Gaussian-based [1, 2], sparse-query [3], voxel-based [4], and tri-perspective [5] methods use different computational structures, yet the paper neither analyzes them nor explains how its six stages would apply to them. More importantly, even a broader set of models would not address the central concern: the paper needs to draw a meaningful insight or direction for future research from the analysis, rather than just report timing measurements.

[1] Y. Huang et al., “GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Prediction,” ECCV, 2024.

[2] J. Park et al., “S2GO: Streaming Sparse Gaussian Occupancy Prediction,” arXiv:2506.05473, 2025.

[3] H. Liu et al., “Fully Sparse 3D Occupancy Prediction,” ECCV, 2024.

[4] Y. Wei et al., “SurroundOcc: Multi-camera 3D Occupancy Prediction for Autonomous Driving,” ICCV, 2023.

[5] Y. Huang et al., “Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction,” CVPR, 2023.

Questions for authors
  1. What CPU or GPU operations account for the large difference in inter-stage residual time between the model variants?

  2. How would the six-stage taxonomy apply to non-BEV representations such as GaussianFormer, SparseOcc, or TPVFormer?

  3. What evidence shows that 10 warm-up iterations and 50 timed passes yield stable medians and p95 values? Could the authors provide per-iteration traces or results with longer runs, especially for configurations near a budget threshold?

Reviewer_FGyBOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a systematic latency analysis for camera-based 3D semantic occupancy prediction. The authors decompose inference into six architectural stages and evaluate six configurations from the BEVDet-OCC and FlashOCC families under a standardized measurement protocol. The study reports median and p95 latency, residual computation time, and runtime variability. Based on these measurements, the paper introduces a budget-aware reporting convention that evaluates occupancy prediction accuracy under explicit inference latency constraints.

Critical strengths
  • Existing occupancy prediction studies often report inference latency under inconsistent hardware and runtime settings, making direct comparisons difficult. Establishing a unified measurement protocol is valuable for fair and reproducible efficiency comparisons across occupancy models.
  • The stage-wise latency decomposition provides useful insights into the computational costs of different architectural components. Reporting median, p95, and residual latency also helps identify performance bottlenecks that may be overlooked by end-to-end measurements alone.
Critical weaknesses
  • Limited novelty of the proposed evaluation framework. The proposed budget-aware evaluation primarily combines existing accuracy metrics with measured inference latency. Although practically useful, the approach resembles conventional accuracy-efficiency trade-off analysis, and its methodological novelty appears limited.
  • Limited architectural generalizability. The experiments are restricted to six configurations from the BEVDet-OCC and FlashOCC families. It remains unclear whether the proposed six-stage taxonomy generalizes to substantially different architectures, such as GaussianFormer[1] and GaussianFormer-2[2], which employ Gaussian-based occupancy representations.
  • Limited validation in real-world deployment settings. All latency measurements are conducted on a server-grade NVIDIA L40 GPU. While the paper emphasizes deployment-aware evaluation, its applicability to resource-constrained platforms remains unclear. Additional experiments on embedded hardware, such as NVIDIA Jetson Orin, would strengthen the practical relevance of the proposed analysis.

[1] Huang, Yuanhui, et al. "Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.

[2] Huang, Yuanhui, et al. "Gaussianformer-2: Probabilistic gaussian superposition for efficient 3d occupancy prediction." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.

Questions for authors
  • What are the key advantages of the proposed budget-aware evaluation over conventional accuracy-latency trade-off analysis?
  • Can the authors extend the evaluation to more diverse occupancy prediction architectures to demonstrate the generalizability of the proposed framework?
  • How would the proposed six-stage taxonomy accommodate alternative occupancy representations, such as Gaussian Splatting-based methods (e.g., GaussianFormer[1] and GaussianFormer-2[2]), that do not follow conventional BEV-based pipelines?
  • Could the authors provide additional latency evaluations on embedded platforms, such as NVIDIA Jetson Orin, to further validate the practical relevance of the proposed analysis for real-world autonomous or embodied systems?
Reviewer_C2oNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents a per-stage latency decomposition of camera-based 3D semantic occupancy inference. Six configurations from the BEVDet-OCC / FlashOCC lineage are evaluated on Occ3D-nuScenes and instrumented into six stages (image backbone, image neck, view transformation, BEV encoder backbone/neck, occupancy head). Timing uses a single idle NVIDIA L40, PyTorch 1.10, FP32, batch size 1, 50 passes over one fixed validation input (after 10 warm-ups), reporting per-stage median and p95 plus an explicit residual. Main findings: the image backbone dominates single-frame latency; view transformation dominates two-frame stereo at R50; the two are co-dominant at Swin-Base; the occupancy head contributes at most 7.1% of end-to-end latency; and BEVDet-OCC vs. FlashOCC differences are concentrated in S4/S6 with roughly constant absolute savings (~16–17 ms) across scales. The paper also proposes a "budget-explicit" evaluation convention: report the highest accuracy among configurations whose measured median latency is below a stated budget.

Critical strengths

S1. Measurement rigor and honesty. The protocol is unusually careful: accumulation over repeated module invocations (Table 6), separate median/p95 reporting, an explicit residual, a synchronisation ablation (Table 12), run-to-run repeats (Table 11), and a distinct-input sensitivity check (Table 13). The paper is precise about what its statistics do and do not establish (e.g., difference-of-medians vs. median-of-differences residual conventions; admission being conditional on the measurement session).

S2. Actionable diagnosis. Separating S4/S5/S6 exposes an offset that grouped reporting hides — FlashOCC's slower neck (−1.11 ms) partially cancelling its S4/S6 savings. The discussion of why percentage speedups are denominator-dependent is a useful service to the community.

Critical weaknesses

W1. Limited novelty of method and findings. Component-level occupancy latency analysis already exists: FlashOCC reports BEV-encoder/head cost separately; FastOcc reports a three-way 2D / 2D-to-3D / 3D split; OneOcc reports four runtime stages on embedded hardware. The six-stage taxonomy here is finer, but the machinery is standard profiling. The "budget-explicit protocol" reduces to selecting the Pareto-optimal accuracy among configurations under a latency constraint — standard practice in efficient-ML and hardware-aware NAS (which the paper itself cites) — and presenting it as a main contribution is overstated.

W2. Headline observations are largely expected. That the 2D backbone and view transformation dominate while the voxel head is a small fraction is strongly implied by previously published figures (e.g., FlashOCC's 7.5→3.1 ms BEV-enc+head at FP16/TensorRT in Table 14). The paper confirms this on one additional platform but adds little conceptual insight beyond it.

Questions for authors

Q1. Would the stage-dominance conclusions hold under FP16/TensorRT or on embedded hardware? Even one configuration re-measured under such settings would substantially strengthen deployment relevance.

Q2. What operations constitute the inter-stage remainder (22–88 ms)? An nsys trace of the two-frame R50 profile would clarify whether it is stereo matching outside S3, fusion ops, data movement, or host-side gaps.

Q3. How exactly are the six stage boundaries mapped onto FlashOCC's modified modules? Are S4–S6 functionally comparable across the two variants (e.g., is the channel-to-height projection inside S6)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.