acceptodds
Under review as a conference paper at ICLR 2027

FIB-SEMBench: A Benchmark for Isotropic Super-Resolution of FIB-SEM Volumes

Abstract

Volumetric electron microscopy, particularly focused ion beam scanning electron microscopy (FIB-SEM), enables three-dimensional visualization of cellular ultrastructure, but acquiring high-resolution isotropic volumes is constrained by high cost and low throughput. 3D super-resolution (SR) offers a way to recover details from low-resolution volumes, while existing SR evaluations rely mainly on image-fidelity metrics. Whether higher image-level fidelity reflects faithful 3D structure or accurate mitochondrial volume estimates remains unclear. To examine this issue, we present FIB-SEMBench, a standardized benchmark for isotropic SR (). The benchmark covers four biologically diverse datasets spanning mouse and Drosophila tissues, with a physics-inspired degradation pipeline and spatially separated data partitioning. We benchmark four 3D SR approaches in terms of image-level fidelity, 3D structural preservation, mitochondrial volume accuracy, and computational efficiency. Our evaluation reveals that higher image-level fidelity does not necessarily correspond to better 3D structure preservation or more accurate mitochondrial volume estimates.

Primary area:
datasets and benchmarks
Keywords:
benchmark, Isotropic Super-Resolution, FIB-SEM

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LsfrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces FIB-SEMBench, a benchmark for evaluating isotropic super-resolution of FIB-SEM volumes, and demonstrates that higher image-level fidelity does not necessarily translate into better 3D structural preservation or more accurate biological quantification. While the research question sounds like interesting, I have several concerns regarding the benchmark construction, particularly its reliance on synthetic degradation without sufficient physical justification or validation against real imaging conditions. Therefore, I am not convinced that the proposed benchmark can faithfully represent the practical challenges of FIB-SEM super-resolution or provide a sufficiently comprehensive and reliable evaluation framework for this field.

Critical strengths

The paper highlights an important limitation of conventional super-resolution evaluation: higher image-level fidelity does not necessarily translate into better 3D structural preservation or downstream biological quantification. Incorporating mitochondrial volume estimation alongside image-quality metrics provides a more application-oriented evaluation perspective, which is potentially valuable for biomedical imaging research.

The paper establishes a unified evaluation protocol covering four biologically diverse FIB-SEM datasets and several representative reconstruction approaches, including CNNs, Transformers, implicit neural representations, and diffusion models. This provides a useful reference for comparing different methods under controlled conditions, although the benchmark's practical reliability and generalizability remain insufficiently validated.

Critical weaknesses
  1. Insufficient physical validation of the degradation model.

The paper claims to employ a physics-inspired degradation pipeline for FIB-SEM super-resolution. However, the degradation process consists primarily of a fixed Gaussian blur ((\sigma=0.8)), 2× downsampling, and generic Poisson and Gaussian noise, without sufficient justification of how these operators and their parameters correspond to the actual FIB-SEM image formation process.

In particular, FIB-SEM resolution is influenced by electron-beam characteristics, electron–sample interactions, scanning conditions, and ion-beam milling. Prior work, such as PolishEM (Fernandez et al., 2020), has demonstrated that effective point spread functions can be estimated directly from FIB-SEM images using Fourier ring correlation.

The authors should clarify whether their Gaussian blur represents an empirically calibrated imaging PSF or merely a heuristic smoothing kernel. Furthermore, it is unclear whether the proposed degradation model captures the distinct physical mechanisms governing lateral imaging resolution and axial sampling.

Most importantly, the authors should examine whether the benchmark conclusions remain consistent under alternative, physically calibrated degradation models. Without such validation, the generalizability of the reported benchmark results to practical FIB-SEM acquisition remains uncertain.

Although the selected OpenOrganelle datasets provide both 4 nm (S0) and 8 nm (S1) pyramid levels, S1 does not constitute an independently acquired low-resolution FIB-SEM measurement. Therefore, agreement between S1 and digitally downsampled S0 cannot validate the physical realism of the proposed blur-and-noise degradation model.

  1. Limited benchmark comprehensiveness and biological validation.

The benchmark evaluates only four methods under a single 2× SR setting. Moreover, biological accuracy is assessed using pretrained segmentation outputs rather than independently validated ground truth.

Questions for authors

1.What is the physical justification for the Gaussian blur and noise parameters, and how can the authors demonstrate that the proposed synthetic degradation accurately reflects real FIB-SEM imaging conditions? I consider this question particularly important, as it directly affects the validity of the entire benchmark and its experimental conclusions. If this issue is resolved satisfactorily, I will consider increasing the rating.

2.How do the authors ensure the reliability of mitochondrial volume measurements obtained from a pretrained segmentation model without independently verified ground-truth annotations?

Reviewer_u7uPOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces FIB-SEMBench, a benchmark for 2× isotropic super-resolution of FIB-SEM volumes, reconstructing 4 nm volumes from synthetically degraded 8 nm inputs. It uses four OpenOrganelle datasets covering mouse and Drosophila tissues, spatially separated training and evaluation regions, and a common training protocol for four types of 3D reconstruction models. Evaluation includes image fidelity, spatial reconstruction errors, mitochondrial segmentation consistency, and computational cost. The main finding is that better image-level scores do not necessarily correspond to better preservation of 3D structure or more accurate mitochondrial volume estimates.

Critical strengths
  1. The benchmark addresses a useful and well-defined problem. Isotropic FIB-SEM super-resolution has a clear practical motivation, and the paper provides a reasonably controlled setup for studying it. The spatial exclusion margins, common degradation process, and shared training protocol are sensible choices. Including four biologically different datasets also makes the comparison more informative than an evaluation on a single tissue.
  2. The emphasis on downstream measurements is valuable. Evaluating reconstructed volumes through mitochondrial segmentation, orthogonal views, and computational cost makes the benchmark more relevant to its intended use. I also appreciate that the paper explicitly acknowledges the limitations of using HR-derived segmentation masks as references. This is a useful foundation for evaluating biological image reconstruction beyond PSNR and SSIM.
Critical weaknesses
  1. The biological evaluation needs stronger validation and internally consistent reporting. Both the HR references and the reconstructed volumes are segmented by MitoNet, so the reported volume errors measure agreement between two sets of model predictions. Appendix G acknowledges this limitation, but the paper does not establish the reliability of the HR masks or explain how the 73 “QC-qualified” samples were selected. This matters when the reported median volume errors are approximately 50–100%: segmentation failures could substantially influence the comparison.
  2. There is also a numerical discrepancy in the central results. Figure 7(a) shows relative volume errors approaching 200,000%, whereas Table A8 reports mean relative errors of 0.730–1.658 for the same 73 samples per method. These values do not appear compatible under Equation A2. Until the discrepancy is resolved, it is difficult to assess the reported correlations and the reliability of the biological comparison.
  3. The phase-bias analysis does not yet establish stride-locked artifacts. The statistic in Section 3.5 measures variation between mean residuals in spatial phase bins. Increasing the period from 2 to 4 to 8 creates more bins with fewer pixels in each bin. The score can therefore increase even when there is no stride-related artifact; for these nested partitions, increasing variation is partly a consequence of how the statistic is constructed. Figure 6 needs an appropriate control before its increasing trend can support the interpretation in Section 4.2.2. This affects one of the paper’s main arguments for why standard image metrics miss systematic structural errors.
Questions for authors
  1. Reference validity: What QC criteria were used to select the 73 biological evaluation samples, and how were the HR MitoNet masks checked? What evidence supports interpreting the measured differences as mitochondrial reconstruction errors rather than sensitivity or failures of the segmentation pipeline?
  2. Numerical consistency: Can you reconcile the relative volume errors in Figure 7(a) with the means in Table A8? Do they use exactly the same samples, masks, units, and error definition? Does resolving this discrepancy change the reported correlations or method rankings?
  3. Artifact interpretation: Does the phase-bias trend remain above an appropriate null that accounts for the increasing number of phase bins? What evidence links the observed patterns to architectural strides rather than to the changing binning of the residuals?
Reviewer_892oOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces FIB-SEMBench, a benchmark for evaluating 3D super-resolution of FIB-SEM images. The benchmark includes four datasets from mouse and fly tissues and compares four deep learning methods for improving image resolution from 8 nm to 4 nm. The evaluation considers image quality, 3D structural consistency, mitochondrial volume accuracy, and computational efficiency. The results show that better image quality scores do not always mean better 3D structure preservation or more accurate biological measurements.

Critical strengths
  1. The paper addresses an important problem in FIB-SEM super-resolution. It shows that traditional image quality metrics may not reflect the preservation of 3D structures and biological information.

  2. The evaluation is comprehensive. Besides traditional image quality metrics, the paper evaluates 3D structural consistency, mitochondrial volume accuracy, and computational efficiency. These additional analyses provide useful insights for future research.

Critical weaknesses
  1. The benchmark only considers 2× super-resolution from 8 nm to 4 nm using synthetic low-resolution images. It is unclear whether the findings can generalize to real low-resolution acquisitions or other resolution settings. This is a major concern from my point of view.

  2. The biological evaluation is limited to mitochondrial volume estimation using one segmentation model. The reported volume errors are also relatively large, which makes me concern about the reliability of the reconstructed images for biological analysis. I think more downstream tasks are needed.

  3. The benchmark only compares four deep learning methods. I think this limits the coverage of existing super-resolution approaches. Including more recent methods and stronger baselines would improve the benchmark.

Questions for authors

Please see the weaknesses mentioned above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.