- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper aims to address the whole image segmentation problem. While SAM prompts a dense grid of points and collects the resulting masks, which might cause overlapped or missing masks, whole image segmentation methods use generative models to generate color images as partition so that every pixel can receive a non-overlapping mask. The question is how to get the colored partition images from the GT masks to train the flow matching model. To avoid the permutation problem, previous work uses centroid of a fixed color field to assign each object a color (UniGS), or supervise the similarity within each object (Gen2seg). This paper proposes to use two fixed color fields, one high frequency and one low frequency, and take their mean within the object mask as the object target color. The low frequency field is to separate large objects and the high frequency field is to separate small objects. To utilize existing object segmentation datasets, this paper introduces mask-conditioning generation during training. At inference, the mask is the whole image. The qualitative and quantitative results show that this paper achieves very strong performance.
Critical strengths
- The qualitative results of across artwork in Figure 1 is very impressive.
- The idea of using two fixed spatial color fields is simple yet effective. Low-frequency map can separate large objects while fall short at small objects. High-frequency map can separate small objects but the large objects receive close-to-grey color.
Critical weaknesses
- This paper uses FLUX2 as backbone while gen2seg uses SD2. It is unclear whether the performance gain mainly come from a newer backbone. This paper lacks an important ablation test. Without this test, I cannot give accept.
- The name "palettes" really confuses me in abstract and introduction. I didn't realize it is a fixed spatial color field until I looked at Figure 2, which is only mentioned in the end of introduction. I thought it was some discrete color set, so I was really confused.
- The fixed spatial color fields are hand-designed. It might need some ablation tests to show the effectiveness of the current design. For example, how about learned spatial embeddings, or other frequency bands, which might perform similarly or better?
Questions for authors
Does the number of palettes matter? What if we use 3 or 4?