- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Brain MRI scans often have multiple problems. The authors use 3 experts, one per problem, and a controller that decides which expert to use, when to stop and when to rollback. The controller is trained w/ SFT & RL. After the controller, a final R³ generic restoration further refine results. Results show that MRI-CARE is competitive on the authors' benchmark.
Critical strengths
-
A real problem and a reasonable way to solve it. The paper writing is generally clear and easy to follow.
-
The designed evaluation show that MRI-CARE is competitive. The paper also checks whether restored scans give better Cortical reconstruction.
Critical weaknesses
- The comparison with baselines is hard to trust, because the baselines and the MRI-CARE are poorly documented and not set up on equal terms. This contains:
- Baselines are 2D methods applied to a 3D problem. PromptIR, InstructIR, TAT-MRI and Restormer are 2D-like. Motion and many other MRI artifacts are 3D problem, and each 2D slice cannot see its neighbors. MRI-CARE works on the whole 3D with pretrained 3D MRI models. A slice-by-slice baseline is therefore at a structural disadvantage. No 3D method is included, and classical methods are also missing.
- Baseline training is also confused. The paper also does not say whether they started from photo-pretrained weights or from scratch, or what lr, batch size, loss were used. Their training differ a lot (PromptIR 6,400 steps, InstructIR 23,296 without validation-based selection, DFPIR and TAT-MRI 36,096, this is very clear in their F.2), so part of the gap may simply be under-training. The authors also admit "These checkpoints have different pretraining and optimization budgets; their comparison does not establish performance under equal training compute." in F.2.
- The proposed method is poorly documented. Only some hyper-parameters are given for the BME-X retrain. Which LLM is used? No code is available at review time. The authors should clarify whether the baseline codes will be released, since they re-train very differently.
- The appendix promises analyses that never appear. Eg in Appendix C, a two-reader study on IXI, but there seems no results at all. I also see no ABIDE cortical reconstruction, but Appendix C " ABIDE cortical reconstruction uses the same software and configuration for all methods". The pattern in appendix makes me unsure, and that weakens trust in the whole paper.
- The gains cannot be credited to the controller. They seem to come from the final R3, which is a strong model the baselines do not have. Without the final R3, the controller alone does not beat baselines on mixed problems (Groups B & C). R³ itself is a all-in-one fixer. It starts from BME-X and is fine-tuned on paired HCP scans. The paper doesn't show results of BME-X alone. The paper doesn't say how often the Ruler asks for refinement. The Ruler's agreement with human is described in the appendix but never reported (it seems they compute AUROC?).
Questions for authors
Questions:
- If time available, what is the results when (a) the R³ refiner applied directly (b) the R³ refiner applied to TAT-MRI's output?
- For baselines, please show full details in appendix. Will the baseline training code be released together with your code?
- Can you add a 3D baseline, or show that the 2D baselines are not disadvantaged on motion?
- Which appendix analyses were actually run? (two-reader study on IXI, ABIDE cortical reconstruction)