- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper learns generative processes for clean images using only noisy measurements with a known noise law. It combines Noise2Noise targets to learn the posterior predictions needed by Gaussian, Poisson, binomial, Gamma, and negative-binomial processes at observed noise levels with self-consistency training to extend those predictions toward clean images; it also proves uniqueness of this continuation under stated conditions. Experiments on simulated CIFAR-10 measurements, photon-counting microscopy, and radar data show improved generation beyond the observed noise level and better capture of spatial uncertainty, though the implementations do not satisfy every theoretical assumption.
Critical strengths
-
Substantive extension beyond Gaussian noise. The paper derives the posterior quantities and noisy training targets needed for several measurement laws, including Gamma and negative-binomial noise, and gives conditions under which self-consistency uniquely determines their continuation. This is a meaningful methodological and theoretical advance over applying existing Gaussian methods to mismatched data.
-
Evidence across controlled and real measurements. Consistency training substantially repairs trajectories that collapse beyond the observed noise level on CIFAR-10 and photon-counting microscopy; the radar results also show better block-level uncertainty coverage than independent pixel sampling. These experiments support the method’s relevance beyond a purely theoretical construction.
Critical weaknesses
The evidence for reliable long-range generation is mixed. In several CIFAR-10 settings, the refined full trajectory performs worse than stopping at the observed level; the radar trajectory also underestimates uncertainty at high information levels. This limits the strength of the paper’s central practical claim.
Questions for authors
Do you observe any empirical pattern that predicts when continuing beyond the observed level improves or worsens sample quality? If so, could it guide the choice of stopping level without clean validation images?
6 comments
Thanks for the addressing
Thank you for the detailed response and the additional experiments. The revised CIFAR-10 results substantially alleviate my concern about the inconsistent benefits of continuation. I also appreciate the additional radar experiments and the clear distinction between agreement with the model’s own posterior and calibration against held-out measurements.
The observation that continuation helps more when the training measurements are noisier is useful. I acknowledge that this does not yet provide a stopping rule without clean validation data, and appreciate that this limitation is now stated explicitly.
Overall, the response strengthens the empirical support for the paper and substantially addresses my main concerns. The remaining limitations do not change my positive assessment, and I will maintain my weak-accept rating.
Dear Reviewer iRAF,
Thank you very much for acknowledging our rebuttal. Is there anything we could clarify that might improve your overall assessment (in particular: what are the weak aspects of the paper that can or perhaps simply cannot be addressed?)
We believe that both our theoretical and experimental results are strong and significant. We are the first (as far as we know) to define a generative process for clean images that works with a variety of noises and is truly useful in realistic imaging applications. Your category ratings support this. The remarks of other reviewers were nearly all resolved by mitigating the PyTorch bug which we discovered very recently (and your question is probably prompted by the same numbers). Fixing (mitigating) the PyTorch bug greatly improved all experimental results (please have a look: now our performance gains are uniform and the presentation is much simpler since we always do the same things). As you noted, the revised experiments provide a much stronger empirical support for the paper. In particular, we have expanded the revised paper to demonstrate that our method is effective on higher-resolution images, such as CelebA, and in settings where the noise is fixed before training rather than resampled at each epoch + we included comparisons with similar methods. Together with demonstrating that our method is robust to variations in hyperparameters, including the number of simulated paths used for the consistency loss, many of the initial limitations have been considerably softened.
We would be happy to address any remaining concerns.
We thank the reviewer for their positive comment. We are happy to answer to their questions below.
W1, part 1: Refined trajectory is worse than stopping at the observed level. We have addressed these concerns in the revised version. We found that the mixed results were partly a consequence of a bug in the GPU-implementation of torch.poisson, which caused the training views for images with simulated Poisson noise to be highly correlated in a certain range of the intensity parameter. These results had motivated us to use the biased averaged estimator instead of the product estimator. We corrected the bug and switched to the product estimator. For the Poisson and negative-binomial models we also strengthened the consistency training, with a larger weight on the consistency loss (Table 9 of the revised paper), and for the negative binomial we extended the continuation. With these changes, continuation improves on or matches early stopping in all settings but one, binomial noise at 180.1 counts, the highest information level for this noise model. Across settings, the gain from continuing shrinks as the information level rises. Since our original choice of estimator was based on results obtained with correlated views, we no longer use the biased averaged estimator in any experiment.
W1, part 2: The radar trajectory underestimates uncertainty at high information levels. In the radar experiment, we supervise the model up to Lobs=1/2, half a look, and the radar data contain no reference beyond about 4.5Lobs. So this is the highest level where we can measure how well our model is calibrated. We reported that at 2Lobs, its 16×16 block-sum coverage is 88.6-89.2% at a nominal 90%, against 82.5-83.2% for independent sampling (Appendix H.5). Beyond that band, the trajectory can only be compared with the network's own posterior, and it realizes 0.78 of the predicted per-pixel spread at 16Lobs and 0.62 at 32Lobs.
We traced this shortfall to the network's posterior, which does not sharpen as looks accumulate. The training targets constrain this sharpening only indirectly, through the consistency loss. We therefore fine-tuned a rerun of the model trained with both the consistency loss and the rollout regression, which before fine-tuning realizes 0.85-0.87 of the predicted spread at 16Lobs and 0.71-0.74 at 32Lobs. The fine-tune runs for 25 epochs with a four-fold consistency weight, without the rollout regression, and with a penalty that keeps the network's output on the observed data, at Lobs and 2Lobs, close to the starting model. It raises the fraction to 0.95-0.99 at 16Lobs and to 0.88-0.95 at 32Lobs, and the block coverage at 2Lobs is 90.0-90.3%. A similar, coarser check against the data at 32Lobs rises from 0.82 and 0.88 to 0.97 on both seeds, against 0.64 and 0.73 for the reported model. The revised paper states the limitation and the fine-tune in the main text and reports the measurements in Appendix H.5.
Q1: About empirical pattern guiding the choice of stopping level. We observe that the noisier the training data, the more continuing beyond the observed level improves on the anchor. On CIFAR-10 the full trajectory improves on the anchor (the predicted posterior mean of the clean image at the observed level) by 9.0, 4.7 and 3.3 FID at 16.1, 178.4 and 613.1 Poisson counts, by 3.0, 3.0 and 0.05 at 8.0, 25.1 and 100.0 binomial counts, and by 5.8, 2.6 and 0.1 at robs=5, 16 and 50 for negative-binomial measurements. At 180.1 binomial counts it is worse than the anchor by 2.2. The information level of the data is known without clean images, so this trend indicates where continuing helps most. It does not tell how far a learned continuation should be run, and we have no rule for choosing the stopping level without clean images. The revised paper states both in its Limitations.