- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Spk2ImgMoE, a mixture-of-experts framework for spike-to-image reconstruction. The idea is that different regions of a spike stream have different spatial, temporal, and noise characteristics, and may therefore need different amounts of computation.
The model uses heterogeneous experts with different computational capacities and introduces bias-driven routing to improve expert utilization without an auxiliary load-balancing loss. Experiments on synthetic SREDS and several real-captured datasets show improvements over existing methods. The authors also study MoE integration, expert heterogeneity, and module placement through ablations.
Critical strengths
-
I like the idea of using heterogeneous experts for this task. Different regions of a spike stream can vary quite a bit in complexity, so using the same amount of computation everywhere may not be ideal. Table 3 provides some good evidence for this design. The heterogeneous configuration reaches 37.76 dB PSNR, outperforming both the all-simple and all-complex configurations while keeping the parameter count moderate. I think this is an interesting result, since it suggests that combining experts with different capacities can be more effective than simply making all experts more complex.
-
The reconstruction results are also quite good, especially considering the evaluation on both synthetic and real-captured datasets. On SREDS, Spk2ImgMoE improves PSNR from 37.46 dB to 37.76 dB over Spk2ImgMamba, with improvements in LPIPS and no-reference image-quality metrics as well. I also appreciate the real-captured experiments, since synthetic data may not fully reflect the characteristics of real spike cameras.
Critical weaknesses
-
Section 3.3 argues that adding a learnable bias to the routing logits automatically compensates for under-utilized experts and prevents expert collapse. However, the formulation (h(x)=W_gx+b) alone does not establish this behavior. A conventional trainable bias is not necessarily updated according to expert utilization, and the paper does not provide a mechanism or theoretical argument showing why under-utilized experts should consistently receive larger bias updates.Table 2 demonstrates an improvement in reconstruction accuracy after adding the bias, but this is not direct evidence of improved load balancing. The paper would benefit from reporting expert selection frequencies, load distributions, or utilization changes during training. A comparison against an explicit load-balancing strategy would also help establish the claimed advantage.This concern matters because bias-driven routing is presented as one of the paper's main technical contributions.
-
Compared with Spk2ImgMamba, the proposed method improves PSNR by 0.30 dB, but the parameter count increases from 0.282M to 0.364M and FLOPs increase from 0.099T to 0.126T.The result is encouraging, but it is difficult to determine how much of the improvement comes from the proposed expert allocation rather than additional model capacity. The homogeneous-expert ablations are helpful, but they do not replace a carefully controlled comparison with a non-MoE backbone at a similar computational budget.Furthermore, the paper does not report practical inference latency or a quantitative uncertainty analysis for the main benchmark comparisons. These would help assess whether the proposed architecture provides a robust efficiency advantage.
Questions for authors
-
Could the authors provide expert selection frequencies or load distributions with and without the learnable bias? I am particularly interested in whether the bias actually improves expert utilization and reduces expert collapse, or mainly makes the gating function more flexible.
-
How does Spk2ImgMoE compare with a parameter-matched or FLOP-matched non-MoE backbone? If such experiments are not available, how can we distinguish the benefit of heterogeneous expert allocation from the effect of increased model capacity?
Thank you for your constructive feedback. Supporting figures are included in the Appendix (Supp. Figs. 6–8); we cite them below.
Q1. Expert utilization and collapse.
We compare three variants: no balancing (no routing bias or auxiliary balancing loss), standard auxiliary balancing (Baseline + MoE), and load-feedback balancing (our full model).
Our routing bias is updated using expert-load feedback. After each optimizer update, we compare each expert's normalized EMA load with the uniform target:
bi(t+1)=bi(t)+ηb(N1−f^i(t)).
Here, f^i(t) is expert i's normalized EMA load, N is the number of experts, and ηb=0.001. The EMA tracks actual Top-2 assignment shares with a decay of 0.95. Underused experts receive positive bias adjustments and overloaded experts negative adjustments, with magnitudes proportional to the load deviation. This gives the bias an explicit load-correction role, rather than merely increasing gating flexibility. The feedback applies even to experts receiving no assignments.
Supp. Fig. 6 shows per-expert utilization at the same MoE block position over the first 40,000 updates. Without either balancing mechanism, routing gradually concentrates on experts 0/1, each approaching 50% of assignment slots, while experts 2/3 become nearly unused. This is consistent with Top-2 expert collapse, where only two of four experts remain active. Both the auxiliary-loss baseline and the full model maintain participation from all four experts. Utilization curves use a 50-update moving average for display.
Supp. Fig. 7 quantifies the load distribution using normalized EMA-load entropy:
H(t)=−lnN∑i=1Nf^i(t)lnf^i(t).
Mean entropy over the first 40,000 updates is 0.5247 without balancing, 0.7576 for the auxiliary-loss baseline, and 0.8293 for the full model. Without balancing, entropy approaches 0.5, consistent with approximately equal use of only two experts. Entropy is computed at each logged update before display smoothing.
Together, the load-dependent bias update, utilization curves, and entropy statistics support the conclusion that bias feedback counteracts persistent expert collapse in the reported run. The auxiliary-loss baseline also avoids sustained collapse, while the full model achieves the highest mean normalized EMA-load entropy.
Q2. Comparison with a matched dense backbone.
We trained a dense non-MoE baseline matched to Spk2ImgMoE at 0.364M parameters and 0.126T FLOPs. The SREDS results are:
At the same parameter and FLOP budget, Spk2ImgMoE achieves 0.52 dB higher PSNR, higher SSIM, and lower LPIPS.
Table 3 and Supp. Fig. 8 provide complementary quantitative and visual comparisons. All Complex Experts uses more parameters and FLOPs than Ours (0.410M vs. 0.364M; 0.137T vs. 0.126T), yet achieves lower PSNR (37.50 vs. 37.76 dB), higher LPIPS (0.0218 vs. 0.0201), and worse NIQE/BRISQUE scores on real data (6.819/18.923 vs. 6.693/15.183).
The close-up in Supp. Fig. 8 shows the complementary reconstruction behavior of the expert architectures. The separately trained complex-only model recovers sharper edges but leaves artifacts in smooth backgrounds. The simple-only model produces smoother backgrounds but preserves edge details less effectively. Ours combines cleaner backgrounds with clearer edges.
Together, these results show that the heterogeneous model outperforms both a dense model with the same parameter and FLOP budget and a larger all-complex model. They support the benefit of allocating different expert capacities to different features.