- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces FIB-SEMBench, a benchmark for evaluating isotropic super-resolution of FIB-SEM volumes, and demonstrates that higher image-level fidelity does not necessarily translate into better 3D structural preservation or more accurate biological quantification. While the research question sounds like interesting, I have several concerns regarding the benchmark construction, particularly its reliance on synthetic degradation without sufficient physical justification or validation against real imaging conditions. Therefore, I am not convinced that the proposed benchmark can faithfully represent the practical challenges of FIB-SEM super-resolution or provide a sufficiently comprehensive and reliable evaluation framework for this field.
Critical strengths
The paper highlights an important limitation of conventional super-resolution evaluation: higher image-level fidelity does not necessarily translate into better 3D structural preservation or downstream biological quantification. Incorporating mitochondrial volume estimation alongside image-quality metrics provides a more application-oriented evaluation perspective, which is potentially valuable for biomedical imaging research.
The paper establishes a unified evaluation protocol covering four biologically diverse FIB-SEM datasets and several representative reconstruction approaches, including CNNs, Transformers, implicit neural representations, and diffusion models. This provides a useful reference for comparing different methods under controlled conditions, although the benchmark's practical reliability and generalizability remain insufficiently validated.
Critical weaknesses
- Insufficient physical validation of the degradation model.
The paper claims to employ a physics-inspired degradation pipeline for FIB-SEM super-resolution. However, the degradation process consists primarily of a fixed Gaussian blur ((\sigma=0.8)), 2× downsampling, and generic Poisson and Gaussian noise, without sufficient justification of how these operators and their parameters correspond to the actual FIB-SEM image formation process.
In particular, FIB-SEM resolution is influenced by electron-beam characteristics, electron–sample interactions, scanning conditions, and ion-beam milling. Prior work, such as PolishEM (Fernandez et al., 2020), has demonstrated that effective point spread functions can be estimated directly from FIB-SEM images using Fourier ring correlation.
The authors should clarify whether their Gaussian blur represents an empirically calibrated imaging PSF or merely a heuristic smoothing kernel. Furthermore, it is unclear whether the proposed degradation model captures the distinct physical mechanisms governing lateral imaging resolution and axial sampling.
Most importantly, the authors should examine whether the benchmark conclusions remain consistent under alternative, physically calibrated degradation models. Without such validation, the generalizability of the reported benchmark results to practical FIB-SEM acquisition remains uncertain.
Although the selected OpenOrganelle datasets provide both 4 nm (S0) and 8 nm (S1) pyramid levels, S1 does not constitute an independently acquired low-resolution FIB-SEM measurement. Therefore, agreement between S1 and digitally downsampled S0 cannot validate the physical realism of the proposed blur-and-noise degradation model.
- Limited benchmark comprehensiveness and biological validation.
The benchmark evaluates only four methods under a single 2× SR setting. Moreover, biological accuracy is assessed using pretrained segmentation outputs rather than independently validated ground truth.
Questions for authors
1.What is the physical justification for the Gaussian blur and noise parameters, and how can the authors demonstrate that the proposed synthetic degradation accurately reflects real FIB-SEM imaging conditions? I consider this question particularly important, as it directly affects the validity of the entire benchmark and its experimental conclusions. If this issue is resolved satisfactorily, I will consider increasing the rating.
2.How do the authors ensure the reliability of mitochondrial volume measurements obtained from a pretrained segmentation model without independently verified ground-truth annotations?