- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper has two parts. The first is a unified remote sensing segmentation model that handles five prompt settings in one network (generic, referring, interactive, intra-image one-to-many and cross-image one-to-many). It is initialised from PSALM and trained on a multi-instruction dataset reorganised from SIOR, iSAID, NWPU-Refer and RRSIS-D. The second is a training-free "prototype evolution" for batch annotation. After an annotator verifies a few predictions, the true-positive, false-negative and false-positive regions are pooled into prototypes and compressed with K-Means under a fixed budget. Positive prototypes are injected into the LLM input and negative ones are used to suppress mask logits. Table 1 compares the unified model with general and RS-specific baselines. Table 2 evaluates prototype evolution on the Vehicle category of SIOR.
Critical strengths
-
The batch-annotation setting is realistic. Reusing what an annotator has already corrected, without updating a large model, matches how mapping work is done, and splitting the verified result into TP, FN and FP regions that feed separate positive and negative paths is a clean design. I am not aware of prior work doing exactly this. The observations that FN prototypes matter more than TP ones (Table 4) and that K-Means is much better than other compression rules (Table 8) are useful.
-
One model covers five prompt settings and is compared with a current set of baselines, both as released and fine-tuned on the same data. The gains on generic segmentation (PQ 62.60 vs 42.81) and on the one-to-many settings are large.
Critical weaknesses
-
Prototype evolution is in the title, but the evidence for it is thin. The only quantitative result in the main text is Table 2, which covers one category, has no variance, and does not say how many exemplars were verified or how they were picked. The gain is +4.81 gIoU and +4.54 mIoU. The abstract promises "progressive" improvement, yet in Table 7 mIoU is 77.06, 76.67, 77.71 and 77.12 as the sequence grows from 50 to 1000. If sequence length counts verified images, more feedback does not help.
-
The method also looks fragile. On Vehicle, a 7:1 allocation gives 32.25 gIoU (Table 4), and importance-based selection gives 18.46 in Table 8, which appears to use the same setting. Both are below the no-memory baseline of 36.43 in Table 2. On Storage Tank the 4:4 allocation drops to 40.95 while 1:7 and 6:2 give 56.54 and 54.53. This does not fit the statement in A.2.5 that the method does not depend on a highly specific parameter value, and these settings appear to be chosen on the evaluated categories.
-
The unified model looks like PSALM fine-tuned on reorganised RS data. A.3.1 says the model follows PSALM and starts from its weights, but PSALM appears only in the zero-shot block of Table 1, and there is no single-task versus multi-task ablation. How the baselines were fine-tuned is not described, and some entries look odd. RemoteSAM drops from 25.67 to 10.74 PQ after fine-tuning and YOLOE from 13.86 to 9.78 cIoU on cross-image. On referring segmentation the model is about 10 points behind RemoteSAM (55.72 / 69.83 vs 65.75 / 78.96).
Minor points:
- Settings are missing in several tables. mIoU is never defined, Tables 5-7 do not say which category they use, and it is not stated whether Table 1 uses the prototype memory.
- Citations are in textual form where parenthetical ones are needed throughout Sections 1 and 2, e.g. "The SAM series Kirillov et al. (2023); Ravi et al. (2024)". Spaces after periods are missing in several places, e.g. "context.These".
- The same mechanism has several names, "representative feedback-driven" and "representative exemplar-driven" prototype evolution, "fixed-budget clustering memory" and "fixed-budget memory fusion".
Questions for authors
- In Table 2, how many exemplars are verified and how are they selected? Is the model the one trained with Vehicle held out, as in A.2.3? Please report mean and standard deviation over exemplar sets and K-Means seeds, and add more categories.
- What does "Seq. Len." count in Table 7? If it is the number of verified images, why is there no improvement from 50 to 1000, and what does the curve look like between 0 and 50?
- With the same verified exemplars, how does prototype evolution compare with a few steps of LoRA fine-tuning, in accuracy and in cost?
- What does PSALM get when fine-tuned on your training split with the same schedule? How were the baselines in the lower block of Table 1 fine-tuned, and why do RemoteSAM and YOLOE get worse?
- Why can a different allocation or compression rule push the result well below the no-memory baseline? Is there any safeguard against the memory hurting predictions?