acceptodds
Withdrawn from ICLR 2027

Decoupled Spiking Mamba for High-Fidelity Neural Radiance Fields

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Neural Radiance Fields (NeRF) built on Artificial Neural Networks (ANNs) dominate 3D reconstruction and rendering but incur a severe energy bottleneck. Spiking Neural Networks (SNNs) replace Multiply-Accumulate (MAC) with sparse Accumulate (AC) operations for low-power 3D representation, yet SNN-based NeRF methods, despite narrowing the metric gap to ANN-based NeRF, still suffer from geometric fragmentation and floater artifacts. Fragmentation arises because NeRF queries a Multi-Layer Perceptron (MLP) independently per point without neighborhood context, and it coincides with unstable spike firing in deep spiking layers; floater artifacts arise from density ambiguity in unconstrained volume rendering, which the limited capacity of SNNs amplifies. To address these issues, this paper proposes DSMamba-NeRF (Decoupled Spiking Mamba for Neural Radiance Fields), an end-to-end high-fidelity spiking architecture. A Decoupled Spiking Mamba (DSMamba) module tackles fragmentation by splitting voxel channels into a global branch that aggregates neighborhood topology via multidirectional state space scanning and a local branch that preserves high-frequency boundaries through 3D spiking convolutions. A Surface-aware spiking Rendering and Refinement (SRF) mechanism replaces unconstrained density with a spiking VolSDF that imposes explicit geometric constraints near the zero level set to suppress floater artifacts. An Adaptive Ternary Spike Neuron (ATSN) with Spike Firing Adaptation (SFA) expands information capacity and modulates the membrane threshold, ensuring stable gradients and an accumulate dominated inference pipeline, with multiplications confined to the compact decoding heads. On synthetic and real-world scenes, DSMamba-NeRF alleviates both defects, attains state-of-the-art rendering metrics among SNN-based NeRF methods, and approaches ANN-based NeRF in visual quality at low energy cost.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Spiking Neural Networks, Neural Radiance Fields, State Space Models, 3D Reconstruction, Low-Power Neural Rendering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_bfVyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DSMamba-NeRF, a spiking neural network framework for high-fidelity neural radiance field reconstruction. The authors identify geometric fragmentation and floater artifacts as two key limitations of existing SNN-based NeRF methods and introduce three components to address them: a decoupled Spiking Mamba module that captures both global geometric structures and local details, a surface-aware rendering mechanism that incorporates SDF constraints, and an adaptive ternary spiking neuron designed to improve training stability. Experiments on synthetic and real-world datasets demonstrate that DSMamba-NeRF achieves better rendering quality than existing SNN-based NeRF methods, particularly for thin structures and object boundaries, while maintaining low estimated computational energy costs. Ablation studies are also conducted to evaluate the effectiveness of the individual components.

Critical strengths

The paper addresses two specific limitations of existing SNN-based NeRF methods, as geometric fragmentation and floater artifacts. The proposed framework combines global-local geometric modeling, SDF-based surface constraints, and adaptive ternary spiking neurons to tackle these issues from different perspectives. These components are reasonably well motivated and form a coherent framework for improving reconstruction quality while preserving the efficiency advantages of SNNs.

The method consistently outperforms existing SNN-based NeRF baselines across synthetic and real-world datasets, with particularly noticeable improvements in thin structures and object boundaries. Beyond standard rendering metrics, the paper provides edge-region evaluations, qualitative comparisons, and ablation studies to support the effectiveness of the proposed approach. The reported energy estimates also suggest a favorable trade-off between rendering quality and computational cost.

Critical weaknesses

The state update in Eq. (1) resembles a conventional spiking recurrence with spike-dependent resetting. It is unclear what benefits specifically come from the Mamba formulation. Comparisons with simpler spiking recurrent or state-space models would help clarify its novelty.

The paper focuses on geometric fragmentation and floater artifacts, but the evaluation mainly relies on image-level metrics such as PSNR, SSIM, and LPIPS. Although qualitative geometry visualizations are provided, quantitative 3D reconstruction metrics are missing. Evaluations using Chamfer Distance, F-score, or surface accuracy would better support the claimed geometric improvements.

Questions for authors
  1. The state update in Eq. (1) appears similar to a conventional spiking recurrence with spike-dependent resetting. What are the specific advantages of the proposed Mamba formulation? Could the authors compare it with simpler spiking recurrent or state-space models to demonstrate its effectiveness?

  2. Since the paper aims to address geometric fragmentation and floater artifacts, could the authors provide quantitative 3D reconstruction metrics, such as Chamfer Distance, F-score, or surface accuracy, to further validate the claimed geometric improvements?

2 comments
AuthorsOpenReview ↗

Response to Reviewer bfVy: Mamba Formulation and Quantitative Geometry Evaluation

Thanks for your efforts in reviewing our paper and your recognition of our well-motivated and coherent framework, the consistent gains over existing SNN-based NeRF methods, and the edge-region evaluation and ablation studies. The response to your questions is given piece by piece as follows. All section, table, and figure numbers refer to the revised manuscript.

W1: The state update in Eq. (1) resembles a conventional spiking recurrence with spike-dependent resetting. It is unclear what benefits specifically come from the Mamba formulation. Comparisons with simpler spiking recurrent or state-space models would help clarify its novelty.

A1: We sincerely thank you for pointing out that the state update in Eq. (1) resembles a conventional spiking recurrence with spike-dependent resetting, and for suggesting comparisons with simpler spiking recurrent or state-space models to clarify the specific benefits and novelty of the Mamba formulation. The state update in Eq. (1) is a ternary spiking recurrence with spike-dependent reset, and our original text did not state clearly what is inherited from Mamba. We have revised the paper in two ways.

First, we clarified the formulation (Section 3.1, Appendix A.3, Appendix A.6.1, and Table 9). DSMamba keeps three elements of the selective state space model: the diagonal state recurrence, the linear scan cost, and a retention coefficient that changes along the sequence. It realizes this coefficient through the reset of the ternary neuron. Writing the reset as a gate gives a_k = α · 1[s_(k−1) = 0], so the retention switches between 0 and α according to the firing at the previous position and therefore follows the input accumulated since the last spike. This is a binary counterpart of the selective transition of Mamba: the gate is driven by the previous firing instead of a learned function of the current token, and the input write has a fixed gain. In return, the state update needs no multiplication beyond the leak, so the scan stays in the accumulate regime. Table 9 lists the correspondence:

ElementMamba (selective SSM)DSMamba scan unit
State per channelvector h_k of size Nscalar potential v_k
Transitionexp(Δ_k A) in (0, 1), learneda_k in {0, α}, learnable leak α
Gate inputcurrent token x_kprevious firing s_(k−1)
Input writegain set by x_kfixed gain
Readoutcontinuousternary spikes
Arithmeticmultiply-accumulateaccumulate and leak

Second, we added the comparison you suggested (Appendix A.6.6, Table 12). We replace the state update with three simpler recurrences under the same six-direction scan on Lego and Ship (T = 2, s = 2, γ = 0.5; means over three independent runs):

  • Stateless removes the carried potential (a_k = 0), so it has no sequence memory.
  • FixedLeak uses a soft reset, so the retention equals α at every position and does not depend on the data. This is a conventional leaky recurrence with a fixed transition and the same parameter count as ours.
  • SpikingRNN adds a dense recurrent matrix W_rec per direction to our unit.
VariantRetention a_kResetLego PSNR↑Lego CD↓Lego [email protected]%↑Ship PSNR↑Ship CD↓Ship [email protected]%↑
Stateless0none30.273.9579.4428.194.6870.45
FixedLeakαsoft30.763.2582.9428.684.1173.88
SpikingRNN{0, α}hard30.902.9884.3328.843.9775.40
DSMamba (ours){0, α}hard31.132.7885.5729.013.7776.62

The results show three things. (1) Sequence memory matters: compared with Stateless, our unit improves PSNR by 0.86 dB on Lego and 0.82 dB on Ship, raises F score by 6.13 and 6.17 points, and lowers CD by 29.6% and 19.4%. (2) The reset gate matters: compared with FixedLeak at the same parameter count, our unit improves PSNR by 0.37 dB and 0.33 dB and raises F score by 2.63 and 2.74 points. (3) A heavier recurrence does not help: SpikingRNN is 0.23 dB and 0.17 dB lower in PSNR and 1.24 and 1.22 points lower in F score than ours, while its recurrent matrix adds a cost that is quadratic in the channel width.

The benefit of the formulation therefore comes from the reset-gated diagonal retention inside the directional scan, obtained at linear cost and with no multiplication beyond the leak. We have also corrected the sentence in Section 3.1 that described the gate as switching "according to the input".

W2: The paper focuses on geometric fragmentation and floater artifacts, but the evaluation mainly relies on image-level metrics such as PSNR, SSIM, and LPIPS. Although qualitative geometry visualizations are provided, quantitative 3D reconstruction metrics are missing. Evaluations using Chamfer Distance, F-score, or surface accuracy would better support the claimed geometric improvements.

A2: Thanks for your advice, that the image-level metrics alone cannot support the geometric claims. We added a quantitative 3D surface evaluation with Chamfer Distance (CD) and F score on Lego and Ship, considering the limited time (Appendix A.6.2, Table 10, and Figure 11).

Protocol: every surface is extracted on a 256³ grid in a common foreground domain (zero level set for the SDF, thresholds fixed before evaluation for density fields), and 100,000 points are sampled uniformly by surface area. CD is the symmetric unsquared Chamfer distance normalized by the ground-truth bounding box diagonal and multiplied by 10³. [email protected]% is the F score at a tolerance of 0.5% of that diagonal, in percent. All values are means over five runs.

SceneMethodPSNR↑SSIM↑LPIPS↓CD↓[email protected]%↑
LegoNeRF (ANN)31.390.9590.0222.4788.14
LegoSpiking NeRF29.370.9420.0404.8367.95
LegoSpik-NeRF (distillation)30.840.9490.0263.6978.20
LegoDSMamba-NeRF31.130.9530.0252.7885.57
ShipNeRF (ANN)29.390.8720.0883.6378.19
ShipSpiking NeRF27.890.8410.1257.2756.62
ShipSpik-NeRF (distillation)28.710.8540.1075.0869.44
ShipDSMamba-NeRF29.010.8630.0973.7776.62

The table is an excerpt of Table 10 in the Appendix, and bold marks the best SNN-based result. Compared with Spik-NeRF, DSMamba-NeRF lowers CD by 24.7% on Lego and 25.8% on Ship and raises F score by 7.37 and 7.18 points. Compared with Spiking NeRF, it lowers CD by 42.4% and 48.1% and raises F score by 17.62 and 20.00 points. The PSNR margins over Spik-NeRF on these two scenes are only 0.29 dB and 0.30 dB, so the surface metrics reveal a larger geometric gain than the image metrics indicate. F score responds to both defects, because floater artifacts lower precision and geometric fragmentation lowers recall. A gap to the ANN-based NeRF remains (2.57 and 1.57 points in F score), and we report it in the table. In the Appendix, the Figure 11 additionally shows the extracted surfaces on Chair, Ship, and Lego.

Q1: The state update in Eq. (1) appears similar to a conventional spiking recurrence with spike-dependent resetting. What are the specific advantages of the proposed Mamba formulation? Could the authors compare it with simpler spiking recurrent or state-space models to demonstrate its effectiveness?

A1: Thanks for the question. Please see our response to W1.

Q2: Since the paper aims to address geometric fragmentation and floater artifacts, could the authors provide quantitative 3D reconstruction metrics, such as Chamfer Distance, F-score, or surface accuracy, to further validate the claimed geometric improvements?

A2: Thanks for the question. Please see our response to W2.

Thanks for your advice again. If you have any other questions, please feel free to let us know.

Reviewer_yWemOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DSMamba-NeRF, a spiking neural rendering framework that addresses geometric fragmentation and floater artifacts. It combines global state-space scanning with local 3D spiking convolutions, SDF-constrained rendering and refinement, and adaptive ternary spike neurons. Experiments on NeRF Synthetic and LLFF show improved rendering quality over prior spiking NeRF methods, with lower estimated energy cost.

Critical strengths
  1. The method directly addresses fragmented structures and floater artifacts through complementary global and local feature modeling and SDF-based surface constraints. The proposed components are clearly connected to the problems they aim to solve.

  2. The improvements are consistent across all 16 scenes in NeRF Synthetic and LLFF, with better PSNR, SSIM, and LPIPS than Spik-NeRF.

  3. The paper provides useful ablations of ATSN, SRF, and DSMamba, along with comparisons of different backbones and channel-splitting ratios, helping clarify the role of each design.

Critical weaknesses
  1. The energy savings are estimated from operation counts using fixed MAC/AC energy costs, excluding memory traffic and volume rendering. Given the use of explicit voxel grids, these omitted costs may be substantial. Measurements on actual hardware are needed to establish the practical energy benefit.

  2. The ANN comparison is limited to the original NeRF. Since the proposed method also changes the feature representation, decoder, and rendering formulation, it is difficult to isolate the contribution of spiking computation. Stronger ANN baselines and a matched ANN counterpart would make the comparison more informative.

  3. Figure 1 is too small to clearly inspect the visual differences, especially in the cropped regions. Making it wider and enlarging the examples would make the fragmentation and floater artifacts much easier to see.

Questions for authors

Could the authors provide quantitative evaluations of 3D geometry, such as depth error or surface reconstruction accuracy, to more directly support the claims of reduced geometric fragmentation, improved topological consistency, and floater suppression?

4 comments
AuthorsOpenReview ↗

Response to Reviewer yWem: Hardware Efficiency, Baseline Comparisons, and Geometry Evaluation

Thanks for your efforts in reviewing our paper and your recognition of the clear connection between our components and the problems they address, the consistent improvements across all 16 scenes, and the ablation studies. The response to your questions is given piece by piece as follows. All section, table, and figure numbers refer to the revised manuscript.

W1: The energy savings are estimated from operation counts using fixed MAC/AC energy costs, excluding memory traffic and volume rendering. Given the use of explicit voxel grids, these omitted costs may be substantial. Measurements on actual hardware are needed to establish the practical energy benefit.

A1: We sincerely thank you for highlighting the practical concern that our operation, the count estimates exclude memory traffic and volume rendering overheads—costs that may be substantial given our explicit voxel grids—and for emphasizing the need for actual hardware measurements. To address this, we added measurements on GPU hardware under one protocol at 400×400, averaged over Chair and Lego (Appendix A.6.3, Table 11). These measurements cover the complete rendering of an image, so both grid access and volume rendering are included.

MethodRendering time, cached (s/image)↓First frame, including preparation (s)↓Peak GPU memory, rendering (GiB)↓GPU energy (J/image)↓
NeRF (ANN)3.18773.1877≈1.95570–640
Spiking NeRF7.78617.7861≈3.451,380–1,580
Spik-NeRF8.76258.7625≈3.901,550–1,780
Spik-NeRF (distillation)8.62058.6205≈3.901,520–1,750
DSMamba-NeRF1.19331.4086≈2.15210–245

With the explicit voxel grid, DSMamba-NeRF uses about 2.15 GiB at rendering, close to NeRF (about 1.95 GiB), and renders an image 2.7 times faster than NeRF and 6.5 to 7.3 times faster than the spiking baselines. Building the voxel grid takes 0.2153 s once per scene. Measured GPU energy per image is 210 to 245 J, against 570 to 640 J for NeRF and 1,380 to 1,780 J for the spiking baselines.

We would like to be clear about the scope of this evidence. A GPU executes spikes as dense tensor operations over T = 2 steps, so these measurements show the practical cost on conventional hardware, where the advantage of DSMamba-NeRF comes mainly from the explicit voxel grid and the compact decoding heads. The estimate in Section 4 follows the standard convention of 3.2 pJ per MAC and 0.1 pJ per AC and describes event-driven hardware. We do not have measurements on neuromorphic hardware, and we acknowledge this as a limitation of the current work.

AuthorsOpenReview ↗

Response to Reviewer yWem: Hardware Efficiency, Baseline Comparisons, and Geometry Evaluation

W2: The ANN comparison is limited to the original NeRF. Since the proposed method also changes the feature representation, decoder, and rendering formulation, it is difficult to isolate the contribution of spiking computation. Stronger ANN baselines and a matched ANN counterpart would make the comparison more informative.

A2: We sincerely thank you for pointing out that because our method alters the feature representation, decoder, and rendering formulation, comparing solely with the original NeRF cannot isolate the effect of spiking computation, and for suggesting matched ANN counterparts and stronger baselines. We would like to explain how the current evidence should be read.

First, the margins we claim are measured inside the spiking regime, against Spiking NeRF and Spik-NeRF, and NeRF serves as the standard ANN reference (Section 4.1).

Second, the ablations isolate each component inside one fixed spiking architecture: Table 3 (binary neuron, then ATSN, then SRF, then DSMamba), Table 12 (recurrence unit), and Table 13 (threshold mechanism and surrogate gradient). For energy, Table 6 splits the per-query cost by operation type: the spiking feature pipeline accounts for 0.022 μJ of the 0.252 μJ, and the multiply-accumulate decoding heads account for the remaining 0.230 μJ, against 1.77 μJ for NeRF.

Third, to test the components on a stronger explicit backbone, we applied them to TensoRF-VM-192 with T = 2 under its 30k iteration schedule (Tables 10 and 11, row TensoRF*). We ran Lego and Ship, considering the limited time.

SceneMethodPSNR↑SSIM↑LPIPS↓CD↓[email protected]%↑
LegoSpik-NeRF (distillation)30.840.9490.0263.6978.20
LegoTensoRF* (spiking, T = 2)30.960.9510.0262.9483.61
LegoDSMamba-NeRF31.130.9530.0252.7885.57
ShipSpik-NeRF (distillation)28.710.8540.1075.0869.44
ShipTensoRF* (spiking, T = 2)28.960.8580.1033.8275.80
ShipDSMamba-NeRF29.010.8630.0973.7776.62

On this backbone the spiking components reach 30.96 dB on Lego and 28.96 dB on Ship, and the variant trains 100 iterations in 3.7940 s and renders an image in 0.6823 s.

AuthorsOpenReview ↗

Response to Reviewer yWem: Hardware Efficiency, Baseline Comparisons, and Geometry Evaluation

W3: Figure 1 is too small to clearly inspect the visual differences, especially in the cropped regions. Making it wider and enlarging the examples would make the fragmentation and floater artifacts much easier to see.

A3: We sincerely thank you for pointing out that Figure 1 is too small to clearly inspect the visual differences, especially the fragmentation and floater artifacts in the cropped regions. We agree that these details require clearer inspection. We have widened Figure 1 and enlarged the cropped regions in the revised manuscript. For the rebuttal phase, we have provided a high-resolution supplementary material collection (included in the uploaded files), containing significantly enlarged versions of the Figure 1 examples. In addition, Figures 6 and 7 in the appendix provide enlarged qualitative comparisons on NeRF Synthetic and LLFF, and the new Figure 11 shows the extracted 3D surfaces on Chair, Ship, and Lego, where fragmentation and floater artifacts can be inspected directly. Thank you for this helpful suggestion.

Q1: Could the authors provide quantitative evaluations of 3D geometry, such as depth error or surface reconstruction accuracy, to more directly support the claims of reduced geometric fragmentation, improved topological consistency, and floater suppression?

A1: We sincerely thank you for suggesting quantitative evaluations of 3D geometry, such as surface reconstruction accuracy, to directly support our claims of reduced geometric fragmentation, improved topological consistency, and floater suppression. To address this, we added a quantitative evaluation of surface reconstruction accuracy using Chamfer Distance (CD) and F-score on \textit{Lego} and \textit{Ship} (Appendix A.6.2, Table 10, and Figure 11).

Protocol: every surface is extracted on a 256³ grid in a common foreground domain (zero level set for the SDF, thresholds fixed before evaluation for density fields), and 100,000 points are sampled uniformly by surface area. CD is the symmetric unsquared Chamfer distance normalized by the ground-truth bounding box diagonal and multiplied by 10³. [email protected]% is the F score at a tolerance of 0.5% of that diagonal, in percent. All values are means over five runs.

SceneMethodPSNR↑SSIM↑LPIPS↓CD↓[email protected]%↑
LegoNeRF (ANN)31.390.9590.0222.4788.14
LegoSpiking NeRF29.370.9420.0404.8367.95
LegoSpik-NeRF (distillation)30.840.9490.0263.6978.20
LegoDSMamba-NeRF31.130.9530.0252.7885.57
ShipNeRF (ANN)29.390.8720.0883.6378.19
ShipSpiking NeRF27.890.8410.1257.2756.62
ShipSpik-NeRF (distillation)28.710.8540.1075.0869.44
ShipDSMamba-NeRF29.010.8630.0973.7776.62

The table is an excerpt from Table 10 in the Appendix, where bold marks the best SNN-based result. Compared with Spik-NeRF, DSMamba-NeRF lowers CD by 24.7% on \textit{Lego} and 25.8% on \textit{Ship} and raises the F-score by 7.37 and 7.18 points. Compared with Spiking NeRF, it lowers CD by 42.4% and 48.1% and raises the F-score by 17.62 and 20.00 points. The F-score responds to both defects, because floater artifacts lower precision and geometric fragmentation lowers recall. The PSNR margins over Spik-NeRF on these scenes are only 0.29 dB and 0.30 dB, so the surface metrics reveal a larger geometric gain than the image metrics indicate. A gap to the ANN-based NeRF remains (2.57 and 1.57 points in F-score). These two metrics measure surface accuracy and completeness.

Thanks for your advice again. If you have any other questions, please feel free to let us know.

AuthorsOpenReview ↗

Response to Reviewer yWem: Hardware Efficiency, Baseline Comparisons, and Geometry Evaluation

We sincerely thank you for this constructive suggestion. Following your advice, we conducted a focused literature search on ANN-based NeRF methods and their extensions incorporating spiking neurons, identifying the closely related studies [1–4]. Among these works, we selected [1] as the most appropriate baseline for a direct quantitative comparison with an ANN-to-SNN conversion approach to neural rendering, as [1] converts a pretrained ANN-based NeRF into a full SNN and employs temporal accumulation to approximate continuous ANN outputs.

We evaluated this baseline, denoted as ANN-SNN NeRF below, alongside our method on synthetic and real-world datasets. As shown in the table, DSMamba-NeRF achieves PSNR values of 30.54 dB and 26.48 dB with only 2 timesteps, exceeding ANN-SNN NeRF at 256 timesteps by 0.14 dB and 0.15 dB, respectively. These results demonstrate that our method maintains rendering quality with substantially fewer timesteps in this comparison.

MethodSynthetic Dataset PSNR (dB) ↑Real-World Dataset PSNR (dB) ↑
NeRF (ANN baseline)31.1126.85
ANN-SNN NeRF (8 timesteps)12.0713.19
ANN-SNN NeRF (128 timesteps)28.8425.96
ANN-SNN NeRF (256 timesteps)30.4026.33
DSMamba-NeRF (2 timesteps)30.5426.48

Regarding the other related studies [2–4], while they offer valuable insights into integrating spiking mechanisms with neural radiance fields, differences in model paradigms, evaluation metrics, and benchmark datasets prevent a direct quantitative comparison in the table above:

[2] and [3] propose hybrid ANN–SNN models primarily aimed at surface geometry reconstruction rather than full-network spiking rendering. Specifically, these approaches retain the original ANN-based NeRF backbone and employ non-linear, non-spike functions to post-process its density-related outputs for sharper geometric boundaries. Furthermore, because their evaluations focus on geometric quality, neither study reports PSNR metrics for novel view synthesis, making a direct quantitative comparison with our rendering results infeasible.

[4] develops a spiking neural radiance field model built upon explicit/factorized grid frameworks (DVGO and TensoRF), which differs substantially from our architectural paradigm. In addition, [4] conducts evaluations on different datasets and experimental splits from those used in our benchmark, which precludes a fair, apple-to-apple quantitative comparison under a unified setting.

We greatly appreciate the reviewer bringing these works to our attention. We will incorporate the quantitative comparison with [1] as well as a comprehensive discussion of [2–4] into the revised manuscript to clearly contextualize our baseline selection and highlight the distinctions between our framework and existing approaches.

References:

[1] Spiking-NeRF: Spiking Neural Network for Energy-Efficient Neural Rendering

[2] Spiking NeRF: Representing the Real-World Geometry by a Discontinuous Representation

[3] Sharpening Your Density Fields: Spiking Neuron Aided Fast Geometry Learning

[4] SpikingNeRF: Making Bio-inspired Neural Networks See through the Real World

Reviewer_BMPpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DSMamba-NeRF, a spiking neural rendering framework combining global/local voxel processing, SDF-based surface refinement, and adaptive ternary neurons to reduce geometric fragmentation and floater artifacts. It reports improved rendering quality over existing spiking NeRF baselines on synthetic and real-world scenes, alongside lower estimated computational energy.

Critical strengths
  1. The method combines complementary mechanisms for improving spiking neural rendering: global/local voxel processing, SDF-based surface constraints, and adaptive ternary firing. I find the focus on fragmented thin structures and floater artifacts well motivated. The component ablations in Tables 3, 7, and 8 provide useful evidence that these mechanisms contribute to rendering quality.

  2. The reported improvements over Spik-NeRF are consistent across all sixteen synthetic and LLFF scenes. The edge-region evaluation is particularly useful given the paper’s motivation: on LLFF, the improvement increases from 0.24 dB in overall PSNR to 1.38 dB within edge regions (Table 2).

Critical weaknesses
  1. Practical efficiency remains insufficiently demonstrated. The energy advantage is based on operation-count estimates that exclude memory traffic and volume rendering. While Figure 5 compares memory usage across backbone variants and Figure 9 reports relative latency for different temporal settings, neither establishes end-to-end efficiency against the main baselines. I would find absolute rendering time and peak GPU memory under matched hardware, resolution, and quality more informative. Without such measurements or deployment evidence, it remains unclear whether the theoretical energy savings translate into a practical advantage.

  2. The ablations do not fully isolate the contribution of several proposed design choices. Tables 3, 7, and 8 provide useful component-level evidence, but the benefits of multidirectional scanning and skip sampling are not separately established. Likewise, the contribution of the dual-threshold surrogate gradient is not isolated from adaptive thresholding within ATSN. These comparisons would help determine which specific mechanisms are necessary for the reported gains.

Questions for authors
  1. Could you report end-to-end rendering time and peak GPU memory against NeRF and Spik-NeRF on the same hardware at matched resolution and comparable rendering quality, including reference-image encoding and voxel construction? If the intended advantage requires neuromorphic hardware, what deployment evidence supports it?

  2. Could you provide controlled ablations of the scan strategy and the dual-threshold surrogate, keeping the remaining architecture and training settings fixed? In particular, how much do these choices contribute beyond a simpler scan and adaptive thresholding alone?

3 comments
AuthorsOpenReview ↗

Response to Reviewer BMPp: Efficiency Evaluation and Controlled Ablations

Thanks for your efforts in reviewing our paper and your recognition of our complementary design, the well-motivated focus on fragmented thin structures and floater artifacts, and the component ablations and edge-region evaluation. The response to your questions is given piece by piece as follows. All section, table, and figure numbers refer to the revised manuscript.

W1: Practical efficiency remains insufficiently demonstrated. The energy advantage is based on operation-count estimates that exclude memory traffic and volume rendering. While Figure 5 compares memory usage across backbone variants and Figure 9 reports relative latency for different temporal settings, neither establishes end-to-end efficiency against the main baselines. I would find absolute rendering time and peak GPU memory under matched hardware, resolution, and quality more informative. Without such measurements or deployment evidence, it remains unclear whether the theoretical energy savings translate into a practical advantage.

A1: We sincerely thank you for raising this important point regarding the necessity of providing absolute rendering time and peak GPU memory under matched hardware, resolution, and quality conditions. We agree that operation-count estimates alone do not fully establish practical efficiency. To address this, we added absolute measurements on GPU against the main baselines under one protocol, on the same hardware and at 400×400 (Appendix A.6.3, Table 11). Values are averages over Chair and Lego, and spiking methods use T=2.

Rendering:

MethodRendering time, cached (s/image)↓First frame, including preparation (s)↓Peak GPU memory, rendering (GiB)↓GPU energy (J/image)↓
NeRF (ANN)3.18773.1877≈1.95570–640
Spiking NeRF7.78617.7861≈3.451,380–1,580
Spik-NeRF8.76258.7625≈3.901,550–1,780
Spik-NeRF (distillation)8.62058.6205≈3.901,520–1,750
DSMamba-NeRF1.19331.4086≈2.15210–245

Training:

Method100 iterations, Chair (s)↓100 iterations, Lego (s)↓100 iterations, Avg. (s)↓Throughput (10³ rays/s)↑Peak GPU memory, training (GiB)↓
NeRF (ANN)5.20095.06255.131719.958.24
Spiking NeRF22.148621.943822.04624.6419.79
Spik-NeRF24.566324.481924.52414.1823.20
Spik-NeRF (distillation)28.005427.900827.95313.6626.41
DSMamba-NeRF14.151814.055414.10367.2615.70

Bold marks the best SNN-based result. The measurements show the following.

  • Rendering. DSMamba-NeRF renders an image in 1.1933 s from the cached voxel grid, which is 2.7 times faster than NeRF and 6.5 to 7.3 times faster than the spiking baselines. Reference-image encoding and voxel construction take 0.2153 s once per scene, so the first frame takes 1.4086 s and the average over 200 views is 1.1944 s. Its peak rendering memory is about 2.15 GiB, close to NeRF (about 1.95 GiB) and below the spiking baselines (about 3.45 to 3.90 GiB).
  • Quality at these settings. On Chair and Lego, DSMamba-NeRF reaches 33.89 dB and 31.13 dB, against 33.37 dB and 30.84 dB for Spik-NeRF and 34.06 dB and 31.39 dB for NeRF (Table 1).
  • Training. DSMamba-NeRF needs 36.0%, 42.5%, and 49.5% less time per 100 iterations than Spiking NeRF, Spik-NeRF, and Spik-NeRF with distillation, and its peak training memory is 20.7%, 32.3%, and 40.6% lower. By the way ANN-based NeRF still trains 2.7 times faster and uses less memory, because GPU training simulates T = 2 spiking steps with surrogate gradients.
  • Measured GPU energy per image follows rendering time: 210 to 245 J for DSMamba-NeRF, 570 to 640 J for NeRF, and 1,380 to 1,780 J for the spiking baselines.

A GPU executes spikes as dense tensor operations over T=2 T=2 steps, so the GPU advantage of DSMamba-NeRF comes mainly from the explicit voxel grid and the compact decoding heads. While physical deployment on neuromorphic hardware is indeed the ultimate test for SNN models, large-scale neuromorphic hardware is not yet readily accessible in our current experimental environment. Therefore, as is standard practice in the current SNN literature, our hardware energy estimates are based on the well-accepted theoretical MAC/AC energy model. And deploying our model on physical neuromorphic hardware (e.g., Loihi) to validate real-world edge performance remains an important and promising avenue for our future work.

AuthorsOpenReview ↗

Response to Reviewer BMPp: Efficiency Evaluation and Controlled Ablations

W2: The ablations do not fully isolate the contribution of several proposed design choices. Tables 3, 7, and 8 provide useful component-level evidence, but the benefits of multidirectional scanning and skip sampling are not separately established. Likewise, the contribution of the dual-threshold surrogate gradient is not isolated from adaptive thresholding within ATSN. These comparisons would help determine which specific mechanisms are necessary for the reported gains.

A2: We appreciate you insightful feedback on the need to more rigorously isolate the specific mechanisms driving the performance gains—namely, separating the effects of multidirectional scanning and skip sampling, and isolating the dual-threshold surrogate gradient from adaptive thresholding. Accordingly, we added two controlled ablations on \textit{Lego} and \textit{Ship}. All variants, including the full model, are trained from scratch with the same loss weights, optimizer, and learning rate schedule, use T=2 and γ=0.5, and change one factor only (Appendix A.6.4). All values are means over three independent runs.

Scan strategy (Appendix A.6.5, Figures 12 to 14). We compare our six-direction scan with three serialization patterns used by visual state space models. ZOrder follows the Morton curve inside each slice. Serpentine traverses each slice row by row and reverses direction on alternate rows. Serpentine4 runs the serpentine path along rows and columns, each in forward and reverse order. All variants share the same ATSN state update and the skip step s = 2.

VariantScan orderPathsDepth scanLego PSNR↑Lego CD↓Lego [email protected]%↑Ship PSNR↑Ship CD↓Ship [email protected]%↑
ZOrderin-slice Morton curve1no30.743.3181.1228.614.3172.13
Serpentinealternating row scan1no30.863.1682.4528.734.1573.45
Serpentine4serpentine along rows and columns, forward and reverse4no30.982.9584.0628.863.9675.08
Ourssix signed axial directions6yes31.132.7885.5729.013.7776.62

Beyond the simpler single-path Serpentine scan, our scan improves PSNR by 0.27 dB on Lego and 0.28 dB on Ship and raises F score by 3.12 and 3.17 points. Relative to ZOrder, the gain is 0.39 dB and 0.40 dB in PSNR and 4.45 and 4.49 points in F score. The four-path planar Serpentine4 narrows the gap to 0.15 dB and to 1.51 and 1.54 points in F score, with a CD that is 6.1% and 5.0% higher than ours. The results indicate that multiple directions help, and that the axis-aligned paths with the front-to-back depth pair add a further gain.

Dual-threshold surrogate versus adaptive thresholding (Appendix A.6.7, Table 13). We cross the threshold mechanism (fixed or SFA) with the surrogate gradient (single-threshold or dual-threshold). Rate is the mean firing rate, Upper is the share of neuron updates above the second threshold, and Grad. is the ratio of the gradient norm at the first spiking layer to that at the last, all measured on Lego.

VariantThresholdSurrogateLego PSNR↑Lego CD↓Lego [email protected]%↑Ship PSNR↑Ship CD↓Ship [email protected]%↑Rate (%)↓Upper (%)Grad.↑
FixedSinglefixedsingle30.683.4379.9128.564.4871.2528.418.60.21
FixedDualfixeddual30.873.1182.5728.764.1273.5827.116.40.46
SFASingleSFAsingle30.992.9384.0128.923.8573.7920.611.20.54
OursSFAdual31.132.7885.5729.013.7776.6219.810.40.82

Beyond adaptive thresholding alone (SFASingle), the dual-threshold surrogate improves PSNR by 0.14 dB on Lego and 0.09 dB on Ship, raises F score by 1.56 and 2.83 points, and lifts the gradient norm ratio from 0.54 to 0.82. Under the fixed threshold, it improves PSNR by 0.19 dB and 0.20 dB. Adaptive thresholding contributes the larger share: SFA improves PSNR by 0.31 dB on Lego and 0.36 dB on Ship with the single-threshold surrogate, and by 0.26 dB and 0.25 dB with the dual-threshold surrogate, while lowering the mean firing rate from 28.4% to 20.6%. Even with SFA, 10.4% to 11.2% of neuron updates lie above the second threshold, which is the region where a single-threshold surrogate passes little gradient. Each mechanism therefore helps under either setting of the other, and together they improve PSNR by 0.45 dB on both scenes and F score by 5.66 and 5.37 points.

AuthorsOpenReview ↗

Response to Reviewer BMPp: Efficiency Evaluation and Controlled Ablations

Q1: Could you report end-to-end rendering time and peak GPU memory against NeRF and Spik-NeRF on the same hardware at matched resolution and comparable rendering quality, including reference-image encoding and voxel construction? If the intended advantage requires neuromorphic hardware, what deployment evidence supports it?

A1: Thanks for the question. Please see our response to W1. In short, including reference-image encoding and voxel construction, DSMamba-NeRF renders the first frame in 1.4086 s against 3.1877 s for NeRF and 8.6205 s to 8.7625 s for Spik-NeRF.

Q2: Could you provide controlled ablations of the scan strategy and the dual-threshold surrogate, keeping the remaining architecture and training settings fixed? In particular, how much do these choices contribute beyond a simpler scan and adaptive thresholding alone?

A2: Thanks for the question. Please see our response to W2.

Thanks for your advice again. If you have any other questions, please feel free to let us know.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Submission 6055: Summary of Revisions and Responses to Reviewers

Submission 6055: Summary for the Area Chair and Reviewers

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and Program Chairs for coordinating the review process, all reviewers for their careful reading and constructive suggestions, and the staff supporting the review process for their time and effort. Your comments have helped us clarify the formulation of DSMamba-NeRF and strengthen its evaluation.

Following our individual responses, we summarize the main concerns, additional evidence, and revisions below to facilitate further discussion. Detailed replies are available in the response threads for Reviewers BMPp, yWem, and bfVy. We would greatly appreciate your further feedback on these responses and any points that may still require clarification.

Summary of Reviews

R-BMPpR-yWemR-bfVyResponse and revision
Strengths noted in the reviews
Well-motivated focus on geometric fragmentation and floater artifacts, with complementary components✓✓✓
Consistent rendering improvements over existing SNN-based NeRF baselines on synthetic and real-world scenes✓✓✓
Useful component ablations supporting the design✓✓✓
Main concerns
Practical rendering costs, including preparation, GPU memory, and energy✓✓Measurements added: absolute GPU costs under a common protocol, with preparation and cached rendering reported separately.
Interpretation and hardware validation of theoretical energy savings✓✓Scope clarified: measured GPU costs and theoretical MAC/AC energy estimates are distinguished; physical neuromorphic deployment remains future work.
Scan strategy and temporal resolution✓Ablations provided: alternative scan patterns compared in Figures 12–14; Figure 9 evaluates T = 1, 2, 4, and 8 and their quality–cost trade-off.
Dual-threshold surrogate versus adaptive thresholding✓Controlled ablation added: threshold adaptation and surrogate choice varied separately.
Relationship to Mamba and benefits over simpler recurrent units✓Formulation clarified and comparisons added: Stateless, FixedLeak, and SpikingRNN controls.
Quantitative evidence for 3D geometry quality✓✓Evaluation added: Chamfer Distance and F-score on Lego and Ship, with extracted-surface visualizations.
Baseline selection and expanded comparisons✓Comparisons added: ANN-SNN NeRF at 8, 128, and 256 timesteps, alongside NeRF and DSMamba-NeRF; spiking components also evaluated on TensoRF-VM-192.
Visibility of qualitative examples✓Presentation revised: wider Figure 1, larger crops, and high-resolution supplementary examples.

Main Responses and Revisions

  • GPU efficiency (BMPp, yWem). On Chair and Lego under the same GPU hardware at 400 × 400, with spiking methods at T = 2, rendering takes 1.1933 s/image cached or 1.4086 s including scene preparation, versus 3.1877 s for NeRF and 7.7861–8.7625 s for spiking baselines. Peak rendering memory is approximately 2.15 GiB and GPU energy is 210–245 J/image. Table 11 also reports training costs. We distinguish measured GPU performance from theoretical neuromorphic energy estimates (Appendix A.6.3).

  • Quantitative geometry (yWem, bfVy). On Lego/Ship, DSMamba-NeRF reduces Chamfer Distance by 24.7%/25.8% and increases F-score by 7.37/7.18 percentage points versus Spik-NeRF (distillation). Results use a common extraction protocol and five-run means; Figure 11 provides surface visualizations (Appendix A.6.2, Table 10).

  • ANN-to-SNN comparison (yWem). We evaluated ANN-SNN NeRF at 8, 128, and 256 timesteps, alongside NeRF and DSMamba-NeRF. Our method reaches 30.54/26.48 dB on synthetic/real-world datasets with 2 timesteps, versus 30.40/26.33 dB for ANN-SNN NeRF at 256 timesteps—gains of 0.14/0.15 dB. We also clarified baseline selection and architectural, metric, and benchmark differences among related methods.

  • Backbone transfer (yWem). On TensoRF-VM-192, our spiking components achieve 30.96/28.96 dB on Lego/Ship with T = 2 and its 30k-iteration schedule (Tables 10–11).

AuthorsOpenReview ↗

# Submission 6055: Summary for the Area Chair and Reviewers

Submission 6055: Summary for the Area Chair and Reviewers

Main Responses and Revisions

  • Mamba formulation and recurrence controls (bfVy). We clarified the reset-gated diagonal recurrence and its relationship to Mamba (Section 3.1, Appendices A.3/A.6.1, Table 9). Comparisons with Stateless, FixedLeak, and SpikingRNN show 0.37/0.33 dB gains over equal-parameter FixedLeak on Lego/Ship (Appendix A.6.6, Table 12).

  • Scan strategy (BMPp). With the same recurrence and s = 2, we compared the six-direction scan with ZOrder, Serpentine, and Serpentine4. Relative to Serpentine, PSNR improves by 0.27/0.28 dB and F-score by 3.12/3.17 percentage points on Lego/Ship (Appendix A.6.5, Figures 12–14).

  • Temporal resolution (BMPp). The original Figure 9 evaluates T = 1, 2, 4, 8 on Flower. T = 2 reaches 27.14 dB / 0.829 SSIM at 1.8× inference cost relative to T = 1; T = 8 reaches 27.25 dB / 0.836 SSIM at 6.9×, supporting the quality–cost balance of T = 2.

  • Threshold and surrogate ablation (BMPp). We separately vary fixed/adaptive thresholds and single/dual-threshold surrogates. With adaptive thresholds, the dual surrogate adds 0.14/0.09 dB and 1.56/2.83 F-score percentage points on Lego/Ship; the gradient norm ratio rises from 0.54 to 0.82 on Lego (Appendix A.6.7, Table 13; three-run means).

  • Visual clarity (yWem). We widened Figure 1, enlarged its crops, and provided high-resolution supplementary examples, alongside the rendering and surface comparisons in Figures 6, 7, and 11.

Where to Find the Revisions

The main manuscript locations for the formulation, evaluation, and ablation studies are summarized below. Figure 9 contains the temporal-resolution ablation from the original manuscript, and Appendix A.6 collects the additional experimental studies. Section, table, and figure numbers below follow the revised manuscript.

RevisionLocation
Mamba correspondence and scan-unit formulationSection 3.1; Appendices A.3 and A.6.1; Table 9; Figure 10
Quantitative 3D surface evaluation and visualizationsAppendix A.6.2; Table 10; Figure 11
GPU rendering and training costsAppendix A.6.3; Table 11
Transfer to TensoRF-VM-192Appendices A.6.2–A.6.3; Tables 10–11
Shared protocol for controlled ablationsAppendix A.6.4
Scan-pattern comparisonsAppendix A.6.5; Figures 12–14
Temporal resolution and its quality–cost trade-off (T = 1, 2, 4, 8)Figure 9
Recurrence-unit comparisonsAppendix A.6.6; Table 12
Adaptive-threshold and surrogate-gradient ablationsAppendix A.6.7; Table 13
Enlarged qualitative examplesFigure 1; supplementary image collection; Figures 6–7 and 11

We would be grateful if the Area Chair and reviewers could revisit these responses when time permits and share any remaining questions or concerns. Your feedback on whether the additional evidence addresses the issues raised, and the Area Chair's guidance on points that would benefit from further discussion, would be very helpful to us. We would be glad to provide further clarification wherever needed. Thank you again for your time, consideration, and help.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.