- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper reformulates motion segmentation around an ordered source–target frame pair: a source pixel should be selected when its scene point moves independently of the camera, rather than whenever the containing object moves somewhere in the video. MoRSe uses a UniQuery4R-initialized joint encoder, a shallow convolutional branch, and a coordinate-query decoder that reads target memory through cross-attention. Camera prediction is an auxiliary training objective and does not directly enter mask prediction. Predictions can be unioned over targets, and a SAM 3 readout can expand regions to whole objects for conventional evaluation. The authors introduce MoRSe-5K, with 42 sequences and 5,431 frames, and report improvements in moving-region accuracy and suppression of camera-induced false positives.
Critical strengths
-
The paper gives a clear operational formulation of local motion: an arm can move while the rest of a person remains still, and the same source image can have different moving regions depending on the target. MoRSe directly predicts these regions without optical flow, point tracks, or a separate mask generator for native region inference. Figures 4 and 6 provide concrete qualitative evidence of local motion and target-dependent responses. The Static setting usefully isolates false positives when only the camera moves. These are useful design and evaluation choices, even though separating scene motion from camera motion is an established problem.
-
The complete system has substantive empirical support. In Table 1, moving-region J&F is 77.98%, compared with 69.64% for GMOS-S, while Static PER/FER decrease from 0.60%/6.82% to 0.0009%/0.75%. The joint accuracy and false-positive results rule out an explanation based on simply predicting empty masks. Importantly, the paper also evaluates whole-object ground truth on the same frames: MoRSe + SAM 3 achieves 79.32% J&F versus 77.93% for GMOS-S. This does not isolate every source of improvement, but the comparison is not confined to region labels that necessarily penalize instance-level methods. Tables 4 and 9 provide useful component and initialization controls, and the paper appropriately notes that the fusion variants are close within the reported seed variation.
Critical weaknesses
-
The advantage over explicit camera-motion compensation is not sufficiently explained. A natural alternative is to estimate optical flow, predict the per-pixel flow that a static scene would exhibit under the camera motion, and segment the residual. This requires a spatially varying flow field, not subtraction of one global shift, because camera translation produces depth-dependent parallax. The paper does compare related approaches: RoMo uses flow-based epipolar residuals, and several scene-flow models estimate motion in world coordinates (Section 2.2 and Table 1). Their weaker results support MoRSe, but an epipolar residual is not the same as subtracting a predicted static-scene flow field. It remains unclear which failures of explicit compensation MoRSe overcomes: inaccurate camera/geometry estimates, errors in optical flow, or the conversion of residuals into masks. A focused analysis of the existing baselines, or a direct compensated-flow comparison where those baselines do not test this alternative, would clarify the added capability and its accuracy–cost trade-off. This is an unresolved comparison question, not evidence that a simpler pipeline already matches MoRSe.
-
Small but genuine motion can be missed entirely, not just segmented with imprecise boundaries. Section 5 and Appendix H acknowledge this failure, and Figure 5 shows partial recovery of a swaying branch and a small animal whose motion is not activated at all. This directly limits the claimed ability to detect local movement independently of camera motion. The examples establish a real failure mode, but not its prevalence. Aggregate scores do not show whether strong static suppression comes with reduced sensitivity to subtle motion, or where the method's reliable operating range ends.
Questions for authors
-
Corresponding to Weakness 1: Which failures of explicit camera-motion compensation does MoRSe overcome? Please first use the existing geometric baselines to analyze performance under camera rotation and depth-dependent parallax. If they do not directly test the compensated-flow alternative, could you compare against observed optical flow minus a predicted static-scene flow field under matched frame/window inputs? Please state how camera motion and geometry are estimated without ground-truth test information; a homography is appropriate only where its assumptions hold, while general 3D scenes require depth-aware compensation. Report moving-region J&F, Static PER/FER, and end-to-end runtime on the same hardware, including flow and geometry estimation, with thresholds selected without test-label tuning. Analysis of an existing baseline that directly addresses this question would suffice; the goal is to understand the advantage, not to assume that the simpler method must win.
-
Corresponding to Weakness 2: Could you report detection recall or moving-region J&F by apparent motion magnitude, including the low-displacement cases illustrated in Figure 5? A small threshold sweep, with Static PER/FER reported alongside recall, would help distinguish an adjustable operating-point trade-off from motion that the model does not reliably detect. Please clarify whether adding nearby targets recovers these cases or whether they remain missed under the window protocol.
Discussion between Authors and Reviewers
Dear Reviewers,
The authors have submitted their rebuttal. Please read it and discuss with the authors via posted messages if necessary.
Note that the author-reviewer discussion phase ends on November 18 at 11:59 pm AoE. It is appreciated if you could share your thoughts as soon as possible, so the authors have enough time to clarify any remaining concerns.
Thank you!
Best, Your AC