- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper focuses a medical segmentation model architecture ViTC-UNet, that conditiona a CNN based UNEt with a frozen ViT backbone like DINO and SAM. The authors integrated a SAM-style 2 way attention blocks in the forzen ViT embeddings with prior tokens. The contributions are first the work combines ViT with UNet withour training or fine-tuning ViT. Second, a single channel, token conditioned output supporting open-vocabulary segmentation. Public datasets are evaluated with a diverse sources of baselines.
Critical strengths
- This work is easy to follow and proved effective on the architecture combinations. Introducing a SAM encoder or a DINO-pretrained ViT to support open-vocabulary segmentation is smart.
- The decoder changes and conditioning on a structure token that supports zero-shot transfer are good for generalizability.
Critical weaknesses
- This architectural combination in general are well studied in the domain, like combining SAM and UNet model architecture, or combine a text encoder and segmentation model, to support multi-modality input, prompt input are well popular in the community.
- Baseline comparison lacks many exisitng works, and comparing to raw baseline nnUNet shall not convincing for the innovation.
Questions for authors
- The baseline comparison with nnUNET setup is not clear, 2D versus 3D, Dice vs IoU, splits. Also other existing SAM-style medical segmentation methods.
- Some baselines are 2D, some are 3D, how is the final score evaluated? 3D metrics, case level?
- How to eval the effectiveness of the text encoder? or prompt encoder?