acceptodds
Under review as a conference paper at ICLR 2027

VEOcc++: Learning Persistent Voxel Feature Maps for Embodied Online Semantic Occupancy Mapping and Offline Refinement

Abstract

Semantic occupancy mapping supports embodied perception during exploration and detailed scene understanding afterward, giving rise to two complementary requirements: online incremental mapping for maintaining an up-to-date global map under causal constraints, and offline global map refinement for improving the completed scene representation with additional computation. We present VEOcc++, a unified voxel-centric framework that connects both stages through a persistent sparse voxel feature map. During exploration, the map expands with newly observed regions and integrates aligned local features through reliability-guided updates, while context-aware decoding produces the current semantic occupancy map. After exploration, cached local features are consolidated as complementary evidence and combined with the completed online representation for multi-scale spatial refinement. On EmbodiedOcc-ScanNet, VEOcc+ achieves 67.12% IoU and 58.40% mIoU, establishing new state-of-the-art performance for online incremental mapping. With the local predictor fixed, our online mapping pipeline outperforms VEOcc fusion by 3.70 IoU and 3.74 mIoU points, demonstrating substantial gains from the proposed global mapping design. Offline global map refinement further improves the completed map to 67.79% IoU and 59.37% mIoU using only the observations acquired during exploration. Zero-shot deployment on a mobile robot further demonstrates transferability to unseen environments, onboard feasibility, and scalability to large-scale indoor mapping.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Semantic Occupancy Prediction, Online Occupancy Mapping, Embodied Scene Understanding, Voxel-based Representation, Persistent Feature Memory, Offline Map Refinement

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_NKUqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VEOcc++ maintains persistent voxel features for online semantic occupancy mapping, then refines the completed map using cached observations. It improves mapping accuracy with the local predictor held fixed and demonstrates embedded deployment, though the specific update design and long-term retention need stronger validation.

Critical strengths
  1. The fixed-predictor comparison is convincing: Table 2 shows gains of 3.70 IoU and 3.74 mIoU points over VEOcc fusion.

  2. The online/offline setup is clear. Refinement reuses existing observations, and Table 6 separates the contributions of different inputs.

  3. The Jetson experiments report latency, cache size, and memory growth, providing useful evidence of practical feasibility.

Critical weaknesses
  1. Mean feature accumulation already reaches 57.49 mIoU versus 58.40 for the full model (Table 5). A matched learned gate without the reliability prior would better isolate the proposed update's contribution.

  2. The updater uses historical average reliability but excludes observation count (Appendix A.3). I'd like repeated-view and conflicting-observation tests to check whether well-supported memory resists incorrect updates.

  3. Mapping depends on external poses, and robot experiments use LiDAR-inertial localization. Pose-noise tests are needed to assess how alignment errors affect accumulated features.

  4. Quantitative accuracy tests remain ScanNet-based, while longer robot sequences are evaluated qualitatively. Revisit accuracy and accuracy-over-time curves would better support the long-term memory claim.

  5. Cached local evidence adds only 0.34 mIoU over probabilities plus map features (Table 6), yet the floor-scale cache occupies 576.50 MiB. A bounded-cache experiment would clarify this trade-off.

Questions for authors

See Weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reliability Guidance, Robustness, and Long-Term Mapping

Thank you for your careful review and constructive suggestions. We have uploaded the revised manuscript, with an overview of the changes provided in our comment “Summary of Changes in the Revised Manuscript.” We address your five concerns below.

1. Reliability prior. We add a capacity-matched control retaining the learned gate and candidate adaptation while removing the reliability prior (Table 5). The full model achieves additional gains of 0.20/0.34 IoU/mIoU points, confirming the contribution of explicit reliability guidance beyond learned gating alone.

2. Repeated and conflicting observations. We conduct a controlled experiment with repeated correct observations followed by conflicting evidence (Appendix C.4, Table 12). The results show that accumulated correct observations lead to lower update weights for subsequent incorrect observations, fewer prediction flips, and smaller degradation in ground-truth class probability. This demonstrates that the learned update mechanism adapts to accumulated evidence and improves memory stability.

3. Pose noise robustness. We evaluate three levels of cumulative pose noise with simulated loop correction and post-run reconstruction (Appendix C.3, Table 9). Both correction strategies substantially recover mapping accuracy, while offline refinement consistently provides additional improvements. In practical robotic systems, localization errors can be continuously mitigated through loop closure and factor-graph optimization. Our results suggest that, with such pose maintenance, accumulated drift need not cause the substantial degradation observed under uncorrected noisy poses.

4. Long-term mapping accuracy. We acknowledge that precise quantitative evaluation of real-world mapping accuracy is difficult without dense ground-truth annotations. To provide quantitative evidence, we add accuracy-over-time analysis with accumulated RGB observations and voxel-level accuracy analysis across observation frequencies (Figs. 7–8). Both exhibit favorable accuracy trends as multi-view observations accumulate, supporting the benefits of persistent feature memory during repeated observations.

5. Local feature cache overhead. We appreciate this suggestion. For the 235 m Laboratory Floor sequence, the complete local feature cache occupies only 576.50 MiB, a modest requirement compared with storage capacities commonly reaching hundreds of GB or several TB. Importantly, the cache is only needed for post-exploration refinement and can be discarded once refinement is completed, avoiding permanent accumulation of cached observations. The complete pipeline also runs successfully on a Jetson Orin 16GB. We agree that bounded-cache experiments could further characterize the storage–accuracy trade-off, although this is a secondary consideration to our central investigation of the benefits of reusing accumulated frame-wise evidence. Given the limited revision time, we prioritized additional experiments addressing memory behavior, robustness, and inference efficiency. If you consider a bounded-cache experiment particularly important to their assessment, please let us know, and we will make every effort to provide it.

We sincerely appreciate your valuable feedback, which has helped strengthen the experimental validation and practical analysis of our framework.

AuthorsOpenReview ↗

Additional Results on Reliability Guidance, Robustness, and Long-Term Mapping

The tables below present the results discussed in our initial response in greater detail. IoU and mIoU are reported in percent.

1. Reliability prior. (Table 5)

The added control retains the learned gate and candidate adaptation while removing the reliability prior.

Online updateIoUmIoU
Learned gate without reliability prior66.9258.06
Full model67.1258.40

The matched gate reaches 66.92/58.06 IoU/mIoU; adding the reliability prior raises this to 67.12/58.40. The gains of 0.20/0.34 IoU/mIoU points reflect the prior's contribution with the learned gate retained.

2. Repeated and conflicting observations. (Appendix C.4, Table 12)

We repeat the same correct observation N times before one incorrect observation. Evaluation uses a shared cohort of 31,449 voxels across 137 validation scenes, correctly classified before the incorrect update under all settings.

Correct repetitions NPrediction flip rate after incorrect update (%) ↓Mean gate for incorrect update ↓Change in ground-truth probability ↑
129.550.4795−0.2122
222.010.4400−0.1756
414.920.3967−0.1291
811.680.3652−0.0995

Increasing correct repetitions from one to eight reduces the flip rate from 29.55% to 11.68% and the mean gate from 0.4795 to 0.3652. The ground-truth class probability change moves closer to zero, from −0.2122 to −0.0995. These results indicate better memory stability under conflicting observations.

3. Pose noise robustness. (Appendix C.3, Table 9)

We simulate cumulative pose noise by injecting translational and yaw drift into the camera poses. Mild, moderate, and severe noise correspond to translational drift of 0.5%, 1.0%, and 2.0% of traveled distance, paired with yaw drift of 0.005°, 0.010°, and 0.020° per meter, respectively. Both correction strategies are also simulated.

Each entry reports IoU/mIoU (%). Bold values in parentheses indicate the gains from offline refinement over online mapping (percentage points).

NoiseUncorrected: OnlineUncorrected: OfflineLoop correction: OnlineLoop correction: OfflinePost-run rebuild: OnlinePost-run rebuild: Offline
Clean67.12/58.4067.79/59.37 (+0.67/+0.97)————
Mild64.65/55.9265.23/56.70 (+0.58/+0.78)65.95/57.2666.57/58.02 (+0.62/+0.76)66.64/57.9267.29/58.79 (+0.65/+0.87)
Moderate59.82/50.9560.20/51.46 (+0.38/+0.51)63.41/54.6363.90/55.14 (+0.49/+0.51)65.60/56.7266.20/57.50 (+0.60/+0.78)
Severe50.83/41.6151.03/41.86 (+0.20/+0.25)57.45/48.4557.68/48.65 (+0.23/+0.20)63.02/53.7863.52/54.40 (+0.50/+0.62)

Both pose correction strategies mitigate the accuracy degradation caused by accumulated pose noise, while offline refinement consistently provides further improvements across all tested noise levels and pose-handling strategies.

4. Long-term mapping accuracy. (Figs. 7–8)

Fig. 7 evaluates accuracy over the observable regions covered by the corresponding fused frames:

Fused RGB framesIoUmIoU
565.0155.80
1065.8955.75
1565.7556.09
2066.2556.94
2566.6757.84
3067.1258.40

At the frame level, fusing more RGB frames generally improves IoU and mIoU, reaching 67.12% IoU and 58.40% mIoU with 30 frames.

Fig. 8(a,b) groups accuracy by voxel observation frequency:

Observation countIoUmIoU
162.2553.33
266.4357.76
368.6059.67
469.8560.84
570.6761.59
671.1662.69
771.5063.44
871.7263.06
971.6363.89
1071.6463.00

At the voxel level, accuracy generally improves with the number of observations. This highlights the benefit of repeated multi-view observations for persistent voxel mapping.

5. Local feature cache overhead.

We have provided a detailed discussion of the local feature cache overhead in our initial response.

AuthorsOpenReview ↗

Official Comment by Authors

Thank you again for your thoughtful and constructive feedback. We have carefully considered your suggestions and made corresponding revisions in both our response and the revised manuscript. We would greatly appreciate hearing whether these updates have adequately addressed your concerns or if any issues remain. We look forward to further discussion and would be happy to provide additional clarification if needed.

Reviewer_u8VjOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses the task of embodied semantic mapping within a unified element framework. Existing methods primarily perform cross-frame fusion at the semantic prediction level, making it difficult to fully preserve historical observation features. The authors propose the Persistent Voxel Feature Map, which achieves cross-frame information fusion through a reliability-guided feature update mechanism. They also use a context-aware decoder to generate online semantic occupancy maps. After the exploration phase, the complete map is further optimized offline by combining the local features of the cache with the global spatial context.

The experimental results in this paper are impressive. The model achieved excellent results on both EmbodiedOcc-ScanNet and Occ-ScanNet, two commonly used datasets for occupancy prediction. The authors also deployed the model to a real mobile robot using a zero-shot approach and validated the practical feasibility of the method in indoor environments at the room and floor scales. This is something that many occupancy prediction methods have not validated. I believe the main contribution of this paper lies in integrating the fusion of persistent voxel features with offline global optimization into a unified framework, and in demonstrating its performance and potential for practical deployment through comprehensive experiments.

Critical strengths

This paper simultaneously addresses online incremental mapping and offline global optimization, which aligns well with the practical needs of embodied intelligent systems to continuously maintain an environmental representation during exploration and to further improve map quality after exploration is complete. The excellent experimental results and comprehensive ablation experiments demonstrate the strengths of the methodology presented in this paper.

In addition, the deployment of real robots has largely enhanced the practical value of the work. Compared to previous work on occupancy prediction, which primarily relies on evaluations using public datasets, this paper further conducts zero-shot deployment on real mobile robots and demonstrates indoor trajectories spanning up to 235 m and floor-level semantic occupancy maps. The authors also provide analyses of runtime, memory usage, and map growth on the Jetson Orin platform. These experiments not only demonstrate the method’s transferability to unknown real-world environments but also provide valuable evidence for its application in practical embodied robotic systems.

Critical weaknesses

Overall, this article doesn't have any serious flaws. There are some areas that could be discussed in greater detail.

  1. the computational and storage overhead associated with long-term map-building still warrants further analysis. The Persistent Voxel Feature Map and the local feature cache continue to grow as the search range and sequence length increase. For longer, larger-scale continuous explorations, there is still room to further validate their computational and storage efficiency.

  2. According to Table 2, offline optimization yielded improvements of only 0.67 IoU and 0.97 mIoU. The additional benefits relative to online mapping remain limited. A more in-depth analysis of which scenes or semantic categories can benefit from more significant improvements through offline optimization will help further demonstrate the value of this module.

Questions for authors

I have no major questions for the authors. The paper is clearly presented, and the experimental results sufficiently support its main claims.

3 comments
AuthorsOpenReview ↗

Response to Long-Term Deployment Efficiency and Offline Refinement Analysis

Thank you again for your positive evaluation and recognition of the practical value of our work. We have uploaded the revised manuscript, with an overview of the changes provided in our comment “Summary of Changes in the Revised Manuscript.” We address your two suggestions below.

1. Long-term computational and storage efficiency. We have optimized Context-Aware Decoding to process only regions affected by current updates, effectively addressing the increasing inference latency while preserving accuracy. The updated results (Table 4, Fig. 9) show stable online latency at approximately 0.63 s/frame as the map grows to nearly one million voxels. We consider the 235 m Laboratory Floor exploration already substantial for our target indoor mapping scenarios. Fig. 9 also shows that process memory grows approximately linearly with the voxel count. Extrapolating this trend suggests that a 16 GB platform could accommodate roughly 3–4 times more voxels during online mapping, although this larger scale has not been directly evaluated. For substantially larger outdoor environments, increasing the current 8 cm voxel size could further reduce storage requirements and extend the supported mapping area.

2. Benefits of offline refinement. We have expanded the category-level analysis in Sec. 4.2, identifying the semantic categories that benefit most from offline refinement and providing a clearer understanding of its class-dependent improvements.

We sincerely appreciate your encouraging assessment and constructive suggestions, which have helped us strengthen the experimental analysis and practical validation of our framework.

AuthorsOpenReview ↗

Additional Results on Long-Term Deployment Efficiency and Offline Refinement Analysis

The tables below present the results discussed in our initial response in greater detail.

1. Long-term computational and storage efficiency. (Table 4)

The optimized Context-Aware Decoding provides particularly pronounced efficiency gains for large-scale maps. The table below compares the original and revised results on the Laboratory Floor sequence (travel distance: 235.15 m; map extent: 57.3 × 15.6 m).

MeasurementOriginal manuscriptRevised manuscriptReduction
Mean online latency (s/frame)0.7640.63417.0%
Total online processing time (s)268.08222.5317.0%
Peak online process RSS (GiB)5.1364.7617.3%

On this floor-scale map, average online latency and total processing time both decrease by approximately 17.0%, while peak process RSS decreases by 7.3%. These results highlight the efficiency benefit of the decoding optimization for large-scale online mapping.

2. Benefits of offline refinement. (Table 2 and Sec. 4.2)

The table includes all 11 semantic categories in the same order as Table 2. Category scores are IoUs (%); changes are in points.

MethodIoUmIoUCeilingFloorWallWindowChairBedSofaTableTVsFurnitureObjects
VEOcc++ (Online)67.1258.4043.0966.8751.9549.1157.8770.4973.9960.0148.4467.2853.34
VEOcc++ (Offline)67.7959.3742.7768.7652.1249.5158.6870.7474.6561.4852.3768.0154.01
Change+0.67+0.97−0.32+1.89+0.17+0.40+0.81+0.25+0.66+1.47+3.93+0.73+0.67

Offline refinement improves 10 of 11 semantic categories. The gains are particularly pronounced for TVs, floors, and tables, with IoU increases of 3.93, 1.89, and 1.47 points, respectively.

Thank you for the detailed responses and additional experiments. My concerns have been sufficiently addressed, and I will maintain my original rating.

Reviewer_N7xvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents VEOcc++, which maintains a persistent voxel feature map for online occupancy mapping and offline refinement. It combines learned feature updates and spatial decoding during exploration, then uses cached local features and a sparse U-Net to refine the completed map.

Critical strengths
  1. The method achieves strong results on global occupancy prediction, with further improvements from offline refinement.

  2. Real-world robotic experiments demonstrate transfer to unseen environments and practical deployment feasibility.

Critical weaknesses
  1. Limited technical novelty. The core contribution is to maintain and update persistent voxel features instead of accumulating semantic predictions. However, persistent scene memory and its updates have already been explored in Gaussian-based methods such as EmbodiedOcc[1] and TGSFormer[2]. The paper should clarify what its memory update contributes beyond the change from Gaussian to voxel representations. Other designs like Context-aware decoding and sparse U-Net refinement are also straightforward and incremental extensions.

  2. Unclear advantage of the online memory update. The paper lacks controlled comparisons showing that its online memory update is more effective than those used by Gaussian-based methods. The voxel-based local predictor already performs better than several Gaussian-based methods (Table 1). The stronger global mapping results may therefore come partly from better local predictions, rather than better online memory update.

  3. Insufficient efficiency comparison. The paper reports its own runtime and memory usage but lacks direct comparisons with VEOcc and Gaussian-based methods under the same conditions. Such comparisons are important given the costs of maintaining voxel features, caching local observations, and processing the growing global map.

References

[1] Yuqi Wu et al. EmbodiedOcc: Embodied 3D Occupancy Prediction for Vision-based Online Scene Understanding. ICCV, 2025.

[2] Rui Qian et al. TGSFormer: Scalable Temporal Gaussian Splatting for Embodied Semantic Scene Completion. CVPR, 2026.

Questions for authors

Please address the weaknesses above.

4 comments
AuthorsOpenReview ↗

Response to Concerns on Technical Novelty, Online Mapping Effectiveness, and Efficiency

Thank you for your constructive feedback. We have uploaded a revised manuscript and provided an overview of the main changes and additional experiments in our comment titled “Summary of Changes in the Revised Manuscript.” We kindly refer you to that summary for an overview of the revisions before addressing your specific concerns below.

1. Technical novelty. We acknowledge that persistent scene memory has been explored in Gaussian-based methods and that context-aware decoding and sparse U-Net refinement build upon established architectures. However, our core contribution lies in establishing a practical two-stage framework that unifies online incremental mapping and offline global refinement through a shared persistent voxel feature representation. As clarified in Secs. 1–2, the reliability-guided update integrates observation quality and historical feature relationships, while the accumulated features are further exploited during offline refinement.

2. Effectiveness of online memory updates. Gaussian-centric and voxel-centric methods follow substantially different representation and update pipelines, making a direct substitution of their fusion mechanisms impractical without extensive architectural changes. We therefore provide complementary evidence through the local-to-global accuracy trends in Appendix C.2 (Table 10). All six Gaussian-based methods exhibit decreases in global IoU compared with their local results, ranging from 1.22 to 10.00 points, whereas VEOcc++ achieves slight improvements of 0.14/0.23 IoU/mIoU points. Although local and global predictions are evaluated on different benchmarks, these trends provide indicative evidence that our feature fusion strategy effectively preserves local prediction quality during global mapping.

3. Efficiency comparison. We add direct comparisons with VEOcc† (Table 3) and Gaussian-based methods (Appendix C.2, Table 11). VEOcc++ achieves higher accuracy with comparable latency and GPU memory usage. Specifically, its online processing requires 0.155 s/frame and 1.741 GiB, compared with 0.162 s/frame and 1.770 GiB for EmbodiedOcc, and 0.142 s/frame and 4.237 GiB for GPOcc (DPT). Furthermore, the updated real-world scaling analysis (Fig. 9) demonstrates stable online latency as the persistent map grows to nearly one million voxels.

We appreciate your suggestions, which helped us clarify the technical contributions and strengthen the experimental comparisons. We hope these revisions address your concerns.

AuthorsOpenReview ↗

Additional Results on Online Mapping Effectiveness and Efficiency

The tables below present the results discussed in our initial response in greater detail. IoU and mIoU are reported in percent.

1. Technical novelty.

We have provided a detailed discussion of the technical novelty in our initial response.

2. Effectiveness of online memory updates. (Appendix C.2, Table 10)

MethodIoU LocalIoU GlobalΔ IoUmIoU LocalmIoU GlobalΔ mIoU
EmbodiedOcc53.9551.52−2.4345.4842.53−2.95
EmbodiedOcc++54.9052.20−2.7046.2043.60−2.60
RoboOcc56.4853.30−3.1847.6744.05−3.62
TGSFormer64.4254.42−10.0054.7345.29−9.44
GPOcc (VGGT)63.1461.41−1.7356.1955.39−0.80
GPOcc++ (VGGT)63.7262.50−1.2257.4157.410.00
VEOcc++66.9867.12+0.1458.1758.40+0.23

Δ = Global − Local.

VEOcc++ reaches 67.12/58.40 IoU/mIoU in global mapping, exceeding its local scores by 0.14/0.23 IoU/mIoU points. All six Gaussian-based methods exhibit decreases in global IoU compared with their local results, ranging from 1.22 to 10.00 points. These results suggest that our fusion effectively preserves local prediction quality during global mapping.

3. Efficiency comparison. (Table 3; Appendix C.2, Table 11)

The following measurements use an RTX 3090 on EmbodiedOcc-ScanNet.

MethodTime (s/frame)GPU memory (GiB)Global IoU / mIoU
VEOcc†0.1301.77463.42 / 54.66
EmbodiedOcc0.1621.77050.75 / 42.44
GPOcc (DPT)0.1424.23757.29 / 51.49
VEOcc++0.1551.74167.12 / 58.40

† denotes VEOcc using the same local predictor as VEOcc++.

VEOcc++ achieves the highest accuracy of 67.12/58.40 IoU/mIoU at 0.155 s/frame and 1.741 GiB. Its inference latency is comparable to that of the Gaussian baselines, with GPU memory usage close to that of EmbodiedOcc and much lower than that of GPOcc (DPT).

AuthorsOpenReview ↗

Official Comment by Authors

Thank you again for your thoughtful and constructive feedback. We have carefully considered your suggestions and made corresponding revisions in both our response and the revised manuscript. We would greatly appreciate hearing whether these updates have adequately addressed your concerns or if any issues remain. We look forward to further discussion and would be happy to provide additional clarification if needed.

Thank you for the detailed responses and additional experiments. These strengthen the evidence for the method’s effectiveness and efficiency, especially the local-to-global trends. I also appreciate the strong results and practical value demonstrated by the real-world robot experiments.

I recognize that the framework is effective, but its core technical novelty over prior work still remains unclear to me. Further improvements from combining online feature-memory updates with offline refinement are reasonably expected. The paper does not sufficiently explain what key technical difficulty this integration resolves or what new insight it provides beyond the performance gains. I therefore view the work mainly as a well-executed integration of established ideas.

Given ICLR’s high standards, I maintain my rating of 2 (weak rejection), mainly due to limited technical novelty. However, I would not strongly oppose acceptance if the AC considers its system and application contributions sufficient.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.