- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ReMoM for motion-guided few-shot video object segmentation. It uses SAM 3 to generate object tracks and introduces three modules for motion matching, temporal enhancement, and interaction-aware context modeling.
Critical strengths
- The paper is well organized, and the motivation is clear.
- The masklet matching framework is interesting and makes good use of SAM3.
- The experimental results are impressive, with consistent improvements on MOVE and A2D-Motion.
Critical weaknesses
- The three modules are based on relatively common techniques, such as feature difference, temporal shuffling, and cross-attention. Their unique contributions could be explained more clearly.
- The method relies heavily on SAM3 for generating candidate masklets, but the impact of masklet quality on the final performance is not well discussed.
- The experiments are generally sufficient, but some additional analysis of the model's motion understanding ability would make the results more convincing.
Questions for authors
- Why does the interaction-aware module only use appearance features from surrounding objects instead of also considering their motion features?
- How sensitive is the method to the quality of the masklets generated by SAM 3?
3 comments
Response to Author
Thank you for your careful responses and we tend to retain the rating
Dear Reviewer fUEU,
We are encouraged by your recognition of our response. We sincerely appreciate your dedicated time and effort in offering invaluable feedback.
Best
Response to Reviewer fUEU (1/1).
W1. Unique contributions of the three proposed modules.
Answer to W1: We would like to clarify that our central contributions are the motion-centric masklet matching formulation and three complementary modules for support-defined motion discrimination, providing a framework for equipping SAM 3 with motion understanding capabilities tailored to M-FSVOS. The feature difference, temporal shuffling, and cross-attention just serve as building blocks within our track-to-way relation encoder, counterfactual temporal enhancement, and interaction-aware contextualization, respectively. In what follows, we clarify the contributions of the three proposed modules, each of which is designed to address a unique challenge.
Track-to-way relation encoder. Since motion categories are defined by support exemplars, our encoder explicitly models the relation between query tracks and support motions, capturing fine-grained motion agreement and discrepancy for support-specific discrimination.
Counterfactual temporal enhancement. This module improves motion understanding by enhancing sensitivity to temporal order. It contrasts the factual query with an appearance-preserving shuffled view, and uses their relation difference to strengthen temporally sensitive motion evidence. By capturing order-sensitive information, this module strengthens motion modeling for support-conditioned matching.
Interaction-aware contextualization. In M-FSVOS, many interaction-dependent motions require context from surrounding objects for discrimination. This module selectively aggregates information from surrounding objects, injecting interaction-relevant context that the target track alone does not contain, and therefore helps resolve the ambiguities in interaction-dependent motions.
Experiments further demonstrate the effectiveness of our formulation and module designs, with ReMoM consistently outperforming SAM 3-based baselines and previous state-of-the-art methods on MOVE and A2D-Motion.
W2 / Q2. Sensitivity to SAM 3 masklet quality.
Answer to W2 / Q2: Thanks for pointing out the insufficient discussion of masklet quality. We conduct two experiments: independently dropping each candidate masklet with probability p to simulate missed tracks, and randomly masking a fraction r of temporal object tokens to simulate missing observations associated with fragmented tracks.
ReMoM retains 60.5 J&F with 15% missed objects and 64.5 with 30% fragmented tracks. These results demonstrate that ReMoM still outperforms the previous state of the art, DMA–VSwin-T (51.1 J&F), supporting its effectiveness and robustness with low masklet quality.
W3. Additional analysis of motion understanding ability.
Answer to W3: Thank you for this suggestion. We conduct additional motion analysis experiments with module ablations to investigate whether the model captures motion cues. Specifically, we compare factual queries with static queries that remove motion variation while retaining appearance cues.
All three configurations perform worse with static queries, supporting that they capture motion cues beyond static appearance.
Q1. Why use only appearance features as contextual keys and values?
Answer to Q1: We provide an ablation on features used as cross-attention keys and values to clarify this design, and the results are shown below. Using only motion features decreases J&F by 3.0 points. These results support our choice of appearance features as contextual keys and values.
We conjecture that this is because the model should focus on the target’s own motion, while surrounding-object appearance provides complementary cues for distinguishing interaction types.