- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper adapts the simulator to the acquisition being reconstructed, during training. A neural posterior over the emitter configuration is trained on simulated frames (OT-relaxed NPE, with per-detection uncertainty), and a second head estimates the shared imaging parameters (PSF, background). A parametric emitter distribution (a flow for the spatial density, a small model for the photon counts, and a binomial for the count), together with a Gaussian over the imaging parameters, is fit to the posterior's predictions on the real acquisition and fed back into the simulator through a defensive mixture whose weight λ is ramped from 0 to 0.95. The specimen structure learned from the whole acquisition is thus fed back into every frame's localization. The fitting objectives are derived from an empirical-Bayes variational bound (Appendix A). Experiments cover the SMLM 2016 Challenge (synthetic, high and low SNR) and three real acquisitions, against DECODE, LiteLoc, SHOT and LUNAR.
Critical strengths
- The problem this paper attacks is real. Per-frame methods discard the fact that all frames image the same specimen, and exploiting that structure across the acquisition has remained open since Cox et al. 2012 could not scale to realistic field sizes.
- The external-calibration ablation gives the clearest evidence for the core claim. With the same network and calibration, learning only the spatial density and the count changes RMSE_ax from 33.4 to 19.3 nm, RMSE_lat from 29.0 to 22.5 nm, and Jaccard from 0.888 to 0.912 (high SNR), and from 47.0 to 41.5 / 57.6 to 48.9 (low SNR). This is a controlled comparison of exactly the proposed change. The variance is larger with three seeds (±2.7, ±4.7), so part of the gain is low-confidence.
- The method is unified: emitters, photophysics, count, PSF and background are all latents fit by the same procedure. The pooling asymmetry (arithmetic for the per-block emitters, logarithmic with the Greenberg correction for the shared PSF) is justified by the Appendix A bound, and I agree with the reasoning.
- Self-calibration extends LUNAR (which learns the PSF only, while the simulator photophysics stay uniform, per Table 1's own classification) to the background and the photophysics. The acquisition-only setting is a useful test of this.
Critical weaknesses
- The structure contribution is not decoupled from self-calibration. The Ours-structure vs Ours-fixed comparison learns both the spatial density and the count, so "structure" is doing double duty. More importantly, LUNAR, which learns the PSF without learning the structure, already nearly matches Ours in the acquisition-only high-SNR setting (Jaccard 0.836 vs 0.830, E3D 0.657 vs 0.653). The current results do not let us tell how much of the acquisition-only gain comes from the learned structure versus from the learned imaging model. Please add (a) density-only and count-only ablations, and (b) the missing cell: uniform structure with learned PSF and background, in the acquisition-only setting.
- The real-data results entangle structure quality with the number of detected localizations. Your method detects far fewer localizations than the others (11.0M vs 22.4M on tubulin; 0.41M vs 1.93M on Nup96). Neither metric directly rewards recovery of the full emitter set: a sparser reconstruction can inflate RSP, and FRC and RSP can disagree, as Nup107 shows, where your method has the best FRC (11.8 nm) and the worst RSP (0.779) at the same time. Without ground truth, the real-data advantage is hard to attribute, and I would like §5.2 (and, ideally, the abstract) to acknowledge this. The RSP reference is the pixel-wise temporal standard deviation rather than the SQUIRREL mean image. Appendix F.2 gives a reasonable justification; the main text does not mention the modification, and a numerical RSP comparison against the standard mean-image reference would show the ranking is not an artifact of this choice.
- I could not find a quantitative evaluation of Contribution 3, the fitted density as reconstruction. q_φ(r) is a flow fit to noisy per-frame predictions and is smoothed by construction, so "smooth filaments and clearly defined rings" rests on visual inspection; a quantitative check is needed to support the accuracy claim. On the Challenge data the true emitter distribution is known, so a Wasserstein distance to the true density or a skeleton comparison would be a natural check and is currently absent.
- The comparison population depends on a threshold value that is only in the code. SNR_min "defines the population the network learns to detect" (Appendix C.1): it sets the target set for the synthetic metrics and the learned detection sensitivity that drives the real-data detection counts. The paper does not state its value (it is in the supplementary config) and does not ablate it.
- Stability of the self-reinforcing loop and a derivation mismatch. The loop rests on "we have not observed a collapse", and the componentwise mixture used in practice (Appendix C.2) deviates from the mixture in Eq. (2) used by the derivation. Does Lemma 1, and the proposal correction in Eq. (6), still hold under the componentwise mixture? The assumptions (fixed specimen, static background, one global PSF, one dye family) also exclude drift, field-dependent aberrations and live cells, which are common in practice and should be discussed, especially since a single global photophysics model over 10⁴ frames is in tension with photobleaching.
Questions for authors
- What is the gain from structure alone, decoupled from self-calibration (uniform structure + learned PSF, acquisition-only setting), and what do the density-only and count-only ablations show?
- What value of SNR_min was used, and how sensitive are the results to it?
- Can you give a quantitative evaluation of q_φ(r) against the known ground-truth distribution on the Challenge data, and report wall-clock and GPU cost for your method and the baselines (100k iterations, batch 128, two posteriors, 200 Sinkhorn iterations per sample, flow matching, 10-step ODE sampling)?
- Why is RMSE_ax 42.4 nm for your method versus 35.6 nm for LUNAR in the acquisition-only high-SNR setting, and does the method hold up with added drift or a decaying background?
3 comments
Response 2/2
Q4 a). Why is RMSE_ax 42.4 nm for your method versus 35.6 nm for LUNAR in the acquisition-only high-SNR setting?
We agree with the reviewer that this result is intriguing, as the ranking is reversed compared to the low SNR setting. We have investigated it, and so far our hypothesis is a difference in PSF model: our blind PSF has no blur term, while LUNAR's PSF model has one. A model error produces a bias that does not shrink with the number of photons. At high SNR, it dominates the error; at low SNR, noise drowns it out. This hypothesis is consistent with our ablation study: with the beads PSF instead of the learnt one, our RMSE_ax drops from 42.4 to 17.6 nm (Table 7).
Unfortunately, we did not have time to redo all our runs with a blur term during the short rebuttal period, but we will update our acquisition-only results in the final version. We warmly thank the reviewer for this question, which pointed us to a limitation of our PSF model. Finally, this does not affect our main claim: the learnt structure still improves the results in this setting (see Q1 a).
Q4 b). Does the method hold up with added drift or a decaying background? Our model accounts for neither. Drift would blur the fitted density, and hence weaken the learnt structure, while a decaying background would bias our static background estimate. More generally, as pointed out by the reviewer, our assumptions also exclude other common effects (photobleaching, field-dependent aberrations, live cells). We believe our framework can be extended to relax these assumptions (e.g. by learning a drift trajectory or a smooth time-dependent background), but we did not have time to test this during the short rebuttal period. We have added these limitations to our Conclusion and Discussion section.
W2 a). Without ground truth, the real-data advantage is hard to attribute We share the reviewer's concern: without ground truth, FRC and RSP are imperfect proxies, and to the best of our knowledge, no metric can yet fully assess reconstructions on real acquisitions. This limitation is already discussed in our Conclusion, and we have clarified it in Section 5.3 (formerly 5.2). Accordingly, we have clarified our claims on real data in the Abstract, Introduction and Conclusion, and the Abstract now acknowledges the absence of ground truth.
W2 b). The RSP reference is the pixel-wise temporal standard deviation rather than the mean image
That is correct. As discussed by Culley et al. (2018), the mean image can be unreliable when it is dominated by background or out-of-focus fluorescence: RSP would then measure agreement with the background rather than with the specimen. As a picture is worth a thousand words, we have added a new Figure 5 (Appendix F), which shows the mean and the standard deviation of each dataset.
For completeness, we report below the RSP against the mean image. The ranking is not preserved, but the results are uninformative, as they reflect agreement with the background rather than with the specimen. We still report them here for completeness:
| Tubulin | Nup107 | Nup96 | |
|---|---|---|---|
| DECODE | 0.743 | 0.321 | 0.619 |
| LiteLoc | 0.492 | 0.290 | 0.447 |
| SHOT | 0.576 | 0.339 | 0.461 |
| LUNAR | 0.491 | 0.263 | 0.476 |
| Ours | 0.344 | 0.278 | 0.497 |
We have clarified the main text accordingly, as requested.
W5. Does Lemma 1, and the proposal correction in Eq. (6), still hold under the componentwise mixture?
For Lemma 1, yes: it holds for any , and , as it does not involve the simulation proposal. For Eq. (6), yes in theory: the componentwise mixture leaves the proposal over the PSF and the background unchanged. In practice, however, we approximate this proposal by (Appendix D.1), which becomes increasingly accurate as gets close to 1. On a similar topic, we have studied the impact of the defensive mixture in our new Table 5, and results do not collapse even without it (). See our answer Q3 b) to reviewer x3e6.
Dear Reviewer,
As the discussion period ends in 24 hours, we would be keen to hear if our responses address your concerns. Following your suggestions, we have added the ablation of the learnt structure decoupled from self-calibration (Table 7), the evaluation of the learnt density against a known ground truth (Section 5.2, Table 2), the value and sensitivity of SNR_min (Table 6), and the training costs of all methods (Table 8). We have also clarified our claims on real data and discussed the limitations of our method in the Conclusion.
If our responses resolve your concerns, we would appreciate it if you could take them into account in your assessment. Thank you again for your consideration.
Response 1/2
We thank the reviewer for their feedback.
Q1 a). What is the gain from structure alone, decoupled from self-calibration We have added a new ablation study in Appendix H, Table 7. As requested, we isolated the effect of learnt vs uniform spatial density, in a setting where the count, the photophysics, the PSF and the background are all learnt. This experiment shows that the benefit of the learnt structure is not tied to the external calibration: in the acquisition-only setting, it still improves E3D from 0.605 to 0.653 at high SNR, and from 0.450 to 0.521 at low SNR. Without it, self-calibration alone falls behind LUNAR at high SNR (0.605 vs 0.657).
Q1 b). What do the density-only and count-only ablations show? They show that the spatial density does most of the work: learning qϕ(r) alone already improves over a uniform density, while learning qϕ(K) alone does not beat a uniform prior p0(K). We detail these results in our answer Q3 a) to reviewer x3e6.
Q2. What value of SNRmin was used, and how sensitive are the results to it?
We use SNRmin = 8 in all experiments, which we now state explicitly in Appendix C. We study its influence in a new Table 6 (Appendix E), reproduced here for convenience (Ours-structure, external calibration):
In short, at high SNR, almost all emitters are above the tested thresholds, and the results are almost independent of SNRmin. At low SNR, SNRmin controls the precision-recall trade-off: higher values increase precision but lower recall, as the network learns to ignore dim emitters. Hence, SNRmin should be adjusted to the SNR of each acquisition.
Q3 a). Can you give a quantitative evaluation of qϕ(r) against the known ground-truth distribution on the Challenge data?
Unfortunately, the 2016 SMLM Challenge do not release the ground-truth structure. We simulated a new acquisition with the same imaging setup and a known density, and assessed the reconstruction of the structure with Wasserstein distances, KL divergences and total variation. We describe everything in a new Section 5.2. In short, our learnt density scored best on all but one metric, showing a better overall capacity on this task than other methods, and suggesting that it goes beyond a simple re-interpretation of classical point clouds. We detail this experiment in our answers to reviewers gngG (Q2 a) and x3e6 (Q2).
Q3 b). Report wall-clock and GPU cost for your method and the baselines We report training times in a new Table 8 (Appendix J). In short, for the same input, our training step is among the fastest, but on large real acquisitions, our training is about an order of magnitude slower. We detail these results in our answers to reviewers x3e6 (Q4 c) and gngG (W2 b).