- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies how decision rules affect downstream task scores in super-resolution evaluation. By fixing SR outputs, observer weights, and score maps while varying the decision rule, the authors show that threshold selection can substantially improve downstream F1 scores and sometimes change method rankings. Based on experiments across remote sensing and several additional domains, the paper proposes four reporting requirements for downstream SR evaluation.
Critical strengths
- Controlled analysis. The paper cleanly isolates the effect of the decision rule by keeping reconstructions and observers fixed. This clearly demonstrates that poor hard-decision scores may partly reflect threshold choice rather than complete loss of discriminative information.
- Broad empirical coverage. The analysis is repeated across multiple SR pipelines, observers, adapted models, and additional domains, showing that decision sensitivity is not limited to a single setting.
Critical weaknesses
- Limited novelty. The main techniques—threshold selection, continuous-score diagnostics, matched references, and bootstrap comparisons—are largely standard. The proposed reporting requirements are useful, but appear more like a consolidation of existing evaluation principles than a substantial new methodology.
- Poor presentation. The paper lacks a clear method/framework overview figure or other essential visualizations that would help readers understand the overall evaluation pipeline. The section organization is also unconventional and fragmented. This is a major factor behind my rejection recommendation.
Questions for authors
- What is the main methodological novelty beyond existing work on threshold selection, calibration, and evaluation sensitivity?
- Given that the experiments span heterogeneous tasks and protocols, what evidence supports the generality of the proposed framework across downstream SR evaluation?