- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies efficient promptable video object segmentation. FastSAM2 replaces global cross-frame memory attention with local convolutional aggregation and uses a shared encoder and decoder for image segmentation. Experiments cover video tracking benchmarks and image segmentation on LVIS. The method is faster with competitive results on short and medium videos, while long-term tracking remains weaker.
Critical strengths
-
CCA is a clear technical contribution that uses local convolutions to aggregate cross-frame evidence without global memory attention.
-
The evaluation covers image segmentation and several video benchmarks with comparisons to relevant baselines.
-
The paper is easy to follow, and the tables and figures are informative.
Critical weaknesses
-
CCA retrieves memory only near the object's current location, leaving long occlusions and distant reappearances without a demonstrated recovery mechanism. Speed gains on shorter sequences do not address this limitation.
-
Recent long-video methods receive limited coverage in the experiments. SAM2Long is cited but omitted from the long-video comparison, leaving the positioning against current long-video solutions unclear.
-
The core CCA design lacks an intuitive diagram, so its key operations are mainly conveyed through equations and text.
Questions for authors
The technical contribution is clear, but the method appears less suited to long and complex video scenarios. For the specific point, please refer to Weaknesses.
2 comments
Thank you again for your time and effort in reviewing our submission.
We would like to kindly follow up on our rebuttal, as the rebuttal period is approaching its deadline. We have carefully addressed each of your concerns point by point in our response, and would greatly appreciate it if you could take a moment to review our rebuttal and share any further thoughts or clarifications you may have.
We would be happy to address any remaining concerns or questions.
Thank you very much for your time and consideration!
Response to Reviewer xy4p
We thank the reviewer for emphasizing long-term re-identification and for suggesting a direct comparison with SAM2Long. We agree that local memory interaction is limited after long absence and distant reappearance.
1. Long-term disappearance and reappearance.(W1)
With L=3 and k=9, CCA has a 25×25 receptive field. It supports local correspondence but not global search. We re-evaluated baselines with one unified LVOS v2 val evaluator (140 videos, 239 objects). The original FastSAM2 score of 72.9 used the standard SAM 2 VOS evaluator. We have reported the shared-evaluator scores in a separate table in the paper.
The shared evaluator scores all four methods on LVOS v2 val (140 videos, 239 objects, 66,056 frames). Each object receives its ground-truth mask at first appearance. J&F is evaluated on subsequent frames except the last. An absent frame scores 1 only if the prediction is empty. The reappearance scores start at the first return after a qualifying absence (86 objects in 62 videos for ≥1 s and 27 objects in 23 videos for ≥5 s, at 6 fps). We have documented the lower reappearance scores for FastSAM2 and identified adaptive global retrieval as future work. See the main-text results (Line 371-374), the unified evaluation and subset table (Line 1298-1319), and the discussion of the reappearance gap (Line 1327-1330).
2. Comparison with SAM2Long.(W2)
We agree that citing SAM2Long without a quantitative comparison leaves the long-video positioning incomplete. We have evaluated the official SAM2Long implementation on LVOS v2 val with the official SAM 2.1 Hiera-B+ checkpoint, 1024×1024 inputs, and the first-appearance ground-truth mask as the only prompt for each object. We used three pathways, an IoU threshold of 0.1, and an uncertainty parameter of 2. We have reported 85.0 on the full set with the shared evaluator, versus 83.0 for SAM 2.1 B+ (reported 95% CI for the difference [0.0, +4.2]). The repository reports 85.2. We have given the 20.5 frames/s result separately because it comes from a bf16 end-to-end run on one 175-frame video with one object. The full-set and subset results are at Line 1309-1319, and the SAM2Long settings and timing are at Line 1320-1326.
3. CCA visualization.(W3)
We have added a simplified diagram to the main paper showing current-frame and memory inputs, temporal mixing, same-position similarity gating, local depthwise propagation, weighting across memory entries, and gated residual fusion. Section 3.3 introduces the diagram (Line 214-215), and the diagram and caption appear at Line 216-232. We have kept the detailed stage equations in the appendix (Line 760-782).
We have positioned CCA as an efficient local temporal-memory operator and identified adaptive global retrieval as a direction for future work (Line 421-425).